RWKV v5: Eagle 7B

Eagle 7B is trained on 1.1 Trillion Tokens across 100+ world languages (70% English, 15% multilang, 15% code).

• Built on the RWKV-v5 architecture (a linear transformer with 10-100x+ lower inference cost)
• Ranks as the world's greenest 7B model (per token)
• Outperforms all 7B class models in multi-lingual benchmarks
• Approaches Falcon (1.5T), LLaMA2 (2T), Mistral (>2T?) level of performance in English evals
• Trade blows with MPT-7B (1T) in English evals
• All while being an "Attention-Free Transformer"(opens in new tab)

Eagle 7B models are provided for free, by Recursal.AI(opens in new tab), for the beta period till end of March 2024

Find out more here(opens in new tab)

rnn