Snowflake: Arctic Instruct

Arctic is a dense-MoE Hybrid transformer architecture pre-trained from scratch by the Snowflake AI Research Team. Arctic combines a 10B dense transformer model with a residual 128x3.66B MoE MLP resulting in 480B total and 17B active parameters chosen using a top-2 gating. To read more about this model's release, click here.

Arctic Instruct dispose d’une fenêtre de contexte de 4 096 jetons.

Arctic Instruct est sorti le 30 avril 2024. Ses connaissances s’arrêtent au 31 janvier 2024.