RWKV(opens in new tab) is an RNN (recurrent neural network) with transformer-level performance. It aims to combine the best of RNNs and transformers - great performance, fast inference, low VRAM, fast training, "infinite" context length, and free sentence embedding.
RWKV-5 is trained on 100+ world languages (70% English, 15% multilang, 15% code).
RWKV 3B models are provided for free, by Recursal.AI, for the beta period. More details here(opens in new tab).
#rnn
Modalities
Context
10K
Released
Dec 10, 2023
Knowledge Cutoff
Jun 2023
Token volume and request traffic to this model over time.
RWKV is an RNN (recurrent neural network) with transformer-level performance. It aims to combine the best of RNNs and transformers - great performance, fast inference, low VRAM, fast training, "infinite" context length, and free sentence embedding. RWKV-5 is trained on 100+ world languages (70% English, 15% multilang, 15% code).
RWKV v5 World 3B has a 10,000 token context window.
RWKV v5 World 3B was released on December 10, 2023. Its knowledge cutoff is June 30, 2023.