Provider Routing

OpenRouter routes requests to the best available providers for your model, given your preferences. By default, requests are load balanced across the top providers to maximize uptime, but you can customize how this works using the provider object in the request body.

Load Balancing

For each model in your request, OpenRouter's default behavior is to load balance requests across providers with the following strategy:

Prioritize providers that have not seen significant outages in the last 10 seconds.
For the stable providers, look at the lowest-cost candidates and select one weighted by price.
Use the remaining providers as fallbacks.

Here's an example. Let's say Provider A is $1/million tokens, Provider B is $2/million, and Provider C is $3/million. Provider B recently saw a few outages.

Your request is 3x more likely to be first routed to Provider A than Provider C.
If Provider A is tried first and fails, then Provider C will be tried next.
If both providers fail, Provider B will be tried last.

Custom Routing

You can set the providers that OpenRouter will prioritize for your request using the order field. The router will prioritize providers in this list, and in this order, for the model you're using. If you don't set this field, the router will load balance across the top providers to maximize uptime.

OpenRouter will try try them one at a time and proceed to other providers if none are operational. If you don't want to allow any other providers, you should disable fallbacks as well.

Here's an example, which will end up skipping over OpenAI (which doesn't host Mixtral), try Together, and then fall back to the normal list of providers on OpenRouter:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "order": [
        "OpenAI",
        "Together"
      ]
    },
  })
});

Here's an example that will end up skipping over OpenAI, try Together, and then fail if Together fails:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "order": [
        "OpenAI",
        "Together"
      ],
      "allow_fallbacks": false
    },
  })
});

Required Parameters (beta)

By default, providers that don't support a given LLM parameter will ignore them. But you can change this and only filter for providers that support the parameters in your request.

For example, to only use providers that support JSON formatting:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "require_parameters": true
    },
    "response_format": {
      "type": "json_object"
    },
  })
});

Data Privacy

Some model providers may log prompts, so we display them with a Data Policy tag on model pages. This is not a definitive source of third party data policies, but represents our best knowledge.

OpenRouter's data policy is managed in your privacy settings. You can disable third party model providers that store inputs for training. Alternatively, you can skip or allow them on a per-request basis:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "data_collection": "deny"
    },
  })
});

Disabling a provider causes the router to skip over it and proceed to the next best one.

Disabling Fallbacks

To guarantee that your request is only served by the top (lowest-cost) provider, you can disable fallbacks.

You can also combine this with the order field from Custom Routing to restrict the providers that OpenRouter will prioritize to just your chosen list.

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "allow_fallbacks": false
    },
  })
});

For a complete list of options, see this JSON schema:

{
  "$ref": "#/definitions/ProviderPreferences",
  "definitions": {
    "ProviderPreferences": {
      "type": "object",
      "properties": {
        "allow_fallbacks": {
          "type": [
            "boolean",
            "null"
          ],
          "description": "Whether to allow backup providers to serve requests\n- true: (default) when the primary provider (or your custom providers in \"order\") is unavailable, use the next best provider.\n- false: use only the primary/custom provider, and return the upstream error if it's unavailable.\n"
        },
        "require_parameters": {
          "type": [
            "boolean",
            "null"
          ],
          "description": "Whether to filter providers to only those that support the parameters you've provided. If this setting is omitted or set to false, then providers will receive only the parameters they support, and ignore the rest."
        },
        "data_collection": {
          "anyOf": [
            {
              "type": "string",
              "enum": [
                "deny",
                "allow"
              ]
            },
            {
              "type": "null"
            }
          ],
          "description": "Data collection setting. If no available model provider meets the requirement, your request will return an error.\n- allow: (default) allow providers which store user data non-transiently and may train on it\n- deny: use only providers which do not collect user data.\n"
        },
        "order": {
          "anyOf": [
            {
              "type": "array",
              "items": {
                "type": "string",
                "enum": [
                  "OpenAI",
                  "Anthropic",
                  "HuggingFace",
                  "Google",
                  "Together",
                  "DeepInfra",
                  "Azure",
                  "Modal",
                  "AnyScale",
                  "Replicate",
                  "Perplexity",
                  "Recursal",
                  "Fireworks",
                  "Mistral",
                  "Groq",
                  "Cohere",
                  "Lepton",
                  "OctoAI",
                  "Novita",
                  "DeepSeek",
                  "Infermatic",
                  "AI21",
                  "Featherless",
                  "Mancer",
                  "Mancer 2",
                  "Lynn 2",
                  "Lynn"
                ]
              }
            },
            {
              "type": "null"
            }
          ],
          "description": "An ordered list of provider names. The router will attempt to use the first provider in the subset of this list that supports your requested model, and fall back to the next if it is unavailable. If no providers are available, the request will fail with an error message."
        }
      },
      "additionalProperties": false
    }
  },
  "$schema": "http://json-schema.org/draft-07/schema#"
}