RWKV is an RNN (recurrent neural network) with transformer-level performance. It aims to combine the best of RNNs and transformers - great performance, fast inference, low VRAM, fast training, "infinite" context length, and free sentence embedding.
RWKV 3B models are provided for free, by Recursal.AI, for the beta period. More details here.