This is the base model variant of the StripedHyena series, developed by Together.
StripedHyena uses a new architecture that competes with traditional Transformers, particularly in long-context data processing. It combines attention mechanisms with gated convolutions for improved speed, efficiency, and scaling. This model marks an advancement in AI architecture for sequence modeling tasks.
Modalities
Context
33K
Released
Dec 9, 2023
Knowledge Cutoff
Jun 2023
Token volume and request traffic to this model over time.
This is the base model variant of the StripedHyena series, developed by Together. StripedHyena uses a new architecture that competes with traditional Transformers, particularly in long-context data processing. It combines attention mechanisms with gated convolutions for improved speed, efficiency, and scaling. This model marks an advancement in AI architecture for sequence modeling tasks.
StripedHyena Hessian 7B (base) has a 32,768 token context window.
StripedHyena Hessian 7B (base) was released on December 9, 2023. Its knowledge cutoff is June 30, 2023.