Provider Routing

OpenRouter routes requests to the best available providers for your model, given your preferences. By default, requests are load balanced across the top providers to maximize uptime, but you can customize how this works using the provider object in the request body.

Load Balancing

For each model in your request, OpenRouter's default behavior is to load balance requests across providers with the following strategy:

  1. Prioritize providers that have not seen significant outages in the last 10 seconds.
  2. For the stable providers, look at the lowest-cost candidates and select one weighted by price.
  3. Use the remaining providers as fallbacks.

Here's an example. Let's say Provider A is $1/million tokens, Provider B is $2/million, and Provider C is $3/million. Provider B recently saw a few outages.

  • Your request is 3x more likely to be first routed to Provider A than Provider C.
  • If Provider A is tried first and fails, then Provider C will be tried next.
  • If both providers fail, Provider B will be tried last.

Custom Routing

You can set the providers that OpenRouter will use for your request using the order field. The router will filter this list to only include providers that are available for the model you're using, and then try one at a time, failing if none are available. If you don't set this field, the router will use the default ordering shown on the model page.

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "order": [
        "Azure",
        "Together"
      ]
    },
  })
});

Required Parameters (beta)

By default, providers that don't support a given LLM parameter will ignore them. But you can change this and only filter for providers that support the parameters in your request.

For example, to only use providers that support JSON formatting:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "require_parameters": true
    },
    "response_format": {
      "type": "json_object"
    },
  })
});

Data Privacy

Some model providers may log prompts, so we display them with a Data Policy tag on model pages. This is not a definitive source of third party data policies, but represents our best knowledge.

OpenRouter's data policy is managed on your Account page. You can disable logging there to also disable third party model providers that store logs for purposes like training. Alternatively, you can skip them on a per-request basis:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "data_collection": "deny"
    },
  })
});

Disabling a provider causes the router to skip over it and proceed to the next best one.

Disabling Fallbacks

To guarantee that your request is only served by the top (lowest-cost) provider, you can disable fallbacks:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Hello"},
    ],
    "provider": {
      "allow_fallbacks": false
    },
  })
});

For a complete list of options, see this JSON schema:

{
  "$ref": "#/definitions/ProviderPreferences",
  "definitions": {
    "ProviderPreferences": {
      "type": "object",
      "properties": {
        "allow_fallbacks": {
          "type": [
            "boolean",
            "null"
          ],
          "description": "Whether to allow backup providers to serve requests\n- true: (default) when the primary provider is unavailable, use the next best provider.\n- false: use only the primary provider, and return the upstream error if it's unavailable.\n"
        },
        "require_parameters": {
          "type": [
            "boolean",
            "null"
          ],
          "description": "Whether to filter providers to only those that support the parameters you've provided. If this setting is omitted or set to false, then providers will receive only the parameters they support, and ignore the rest."
        },
        "data_collection": {
          "anyOf": [
            {
              "type": "string",
              "enum": [
                "deny",
                "allow"
              ]
            },
            {
              "type": "null"
            }
          ],
          "description": "Data collection setting. If no available model provider meets the requirement, your request will return an error.\n- allow: (default) allow providers which store user data non-transiently and may train on it\n- deny: use only providers which do not collect user data.\n"
        },
        "order": {
          "anyOf": [
            {
              "type": "array",
              "items": {
                "type": "string",
                "enum": [
                  "OpenAI",
                  "Anthropic",
                  "HuggingFace",
                  "Google",
                  "Together",
                  "DeepInfra",
                  "Azure",
                  "Modal",
                  "AnyScale",
                  "Replicate",
                  "Perplexity",
                  "Recursal",
                  "Fireworks",
                  "Mistral",
                  "Groq",
                  "Cohere",
                  "Lepton",
                  "OctoAI",
                  "Novita",
                  "DeepSeek",
                  "Mancer",
                  "Mancer 2",
                  "Lynn 2",
                  "Lynn"
                ]
              }
            },
            {
              "type": "null"
            }
          ],
          "description": "An ordered list of provider names. The router will attempt to use the first provider in the subset of this list that supports your requested model, and fall back to the next if it is unavailable. If no providers are available, the request will fail with an error message."
        }
      },
      "additionalProperties": false
    }
  },
  "$schema": "http://json-schema.org/draft-07/schema#"
}