Meta: Llama 3.1 405B Instruct (nitro)

meta-llama/llama-3.1-405b-instruct:nitro

Created Jul 238,000 context

$17.9/M input tokens$17.9/M output tokens

The highly anticipated 400B class of Llama3 is here! Clocking in at 128k context with impressive eval scores, the Meta AI team continues to push the frontier of open-source LLMs.

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 405B instruct-tuned version is optimized for high quality dialogue usecases.

It has demonstrated strong performance compared to leading closed-source models in human evaluations.

Usage of this model is subject to Meta's Acceptable Use Policy.

These are higher-throughput endpoints for Llama 3.1 405B Instruct. They may have higher prices.

Sample code and API for Llama 3.1 405B Instruct (nitro)

OpenRouter normalizes requests and responses across providers for you.

To get started, you can use Llama 3.1 405B Instruct (nitro) via API like this:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "meta-llama/llama-3.1-405b-instruct:nitro",
    "messages": [
      {
        "role": "user",
        "content": "What is the meaning of life?"
      }
    ]
  })
});

You can also use OpenRouter with OpenAI's client API:

import OpenAI from "openai"

const openai = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: $OPENROUTER_API_KEY,
  defaultHeaders: {
    "HTTP-Referer": $YOUR_SITE_URL, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": $YOUR_SITE_NAME, // Optional. Shows in rankings on openrouter.ai.
  }
})

async function main() {
  const completion = await openai.chat.completions.create({
    model: "meta-llama/llama-3.1-405b-instruct:nitro",
    messages: [
      {
        "role": "user",
        "content": "What is the meaning of life?"
      }
    ]
  })

  console.log(completion.choices[0].message)
}
main()

See the Request docs for all possible parameters, and Parameters for recommended values.