Provider Routing
OpenRouter routes requests to the best available providers for your model, given your preferences. By default, requests are load balanced across the top providers to maximize uptime, but you can customize how this works using the provider
object in the request body.
Load Balancing
For each model in your request, OpenRouter's default behavior is to load balance requests across providers with the following strategy:
- Prioritize providers that have not seen significant outages in the last 10 seconds.
- For the stable providers, look at the lowest-cost candidates and select one weighted by price.
- Use the remaining providers as fallbacks.
Here's an example. Let's say Provider A is $1/million tokens, Provider B is $2/million, and Provider C is $3/million. Provider B recently saw a few outages.
- Your request is 3x more likely to be first routed to Provider A than Provider C.
- If Provider A is tried first and fails, then Provider C will be tried next.
- If both providers fail, Provider B will be tried last.
Custom Routing
You can set the providers that OpenRouter will prioritize for your request using the order
field. The router will prioritize providers in this list, and in this order, for the model you're using. If you don't set this field, the router will load balance across the top providers to maximize uptime.
OpenRouter will try try them one at a time and proceed to other providers if none are operational. If you don't want to allow any other providers, you should disable fallbacks as well.
Here's an example, which will end up skipping over OpenAI (which doesn't host Mixtral), try Together, and then fall back to the normal list of providers on OpenRouter:
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "mistralai/mixtral-8x7b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"order": [
"OpenAI",
"Together"
]
},
})
});
Here's an example that will end up skipping over OpenAI, try Together, and then fail if Together fails:
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "mistralai/mixtral-8x7b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"order": [
"OpenAI",
"Together"
],
"allow_fallbacks": false
},
})
});
Required Parameters (beta)
By default, providers that don't support a given LLM parameter will ignore them. But you can change this and only filter for providers that support the parameters in your request.
For example, to only use providers that support JSON formatting:
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "mistralai/mixtral-8x7b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"require_parameters": true
},
"response_format": {
"type": "json_object"
},
})
});
Tool Use (beta)
When you send a request with tools
or tool_choice
, OpenRouter will only route to providers that natively support tool use.
Data Privacy
Some model providers may log prompts, so we display them with a Data Policy tag on model pages. This is not a definitive source of third party data policies, but represents our best knowledge.
OpenRouter's data policy is managed in your privacy settings. You can disable third party model providers that store inputs for training. Alternatively, you can skip or allow them on a per-request basis:
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "mistralai/mixtral-8x7b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"data_collection": "deny"
},
})
});
Disabling a provider causes the router to skip over it and proceed to the next best one.
Disabling Fallbacks
To guarantee that your request is only served by the top (lowest-cost) provider, you can disable fallbacks.
You can also combine this with the order
field from Custom Routing to restrict the providers that OpenRouter will prioritize to just your chosen list.
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "mistralai/mixtral-8x7b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"allow_fallbacks": false
},
})
});
Ignoring Providers
Ignore Providers for a Request
You can ignore providers for a request by setting the ignore
field in the provider
object.
Here's an example that will ignore Azure for a request calling GPT-4 Omni:
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "openai/gpt-4o",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"ignore": [
"Azure"
]
},
})
});
Ignore Providers for Account-Wide Requests
You can ignore providers for all account requests by configuring your preferences. This configuration applies to all API requests and chatroom messages.
Warning: Ignoring multiple providers may significantly reduce fallback options and limit request recovery.
When you ignore providers for a request, the list of ignored providers is merged with your account-wide ignored providers.
Quantization
Quantization reduces model size and computational requirements while aiming to preserve performance. However, quantized models may exhibit degraded performance for certain prompts, depending on the method used.
Providers can support various quantization levels for open-weight models.
Quantization Levels
By default, requests are load-balanced across all available providers, ordered by price. To filter providers by quantization level, specify the quantizations
field in the provider
parameter with the following values:
int4
: Integer (4 bit)int8
: Integer (8 bit)fp8
: Floating point (8 bit)fp16
: Floating point (16 bit)bf16
: Brain floating point (16 bit)unknown
: Unknown
Example Request with Quantization
fetch("https://openrouter.ai/api/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${OPENROUTER_API_KEY}`,
"HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
"X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
"Content-Type": "application/json"
},
body: JSON.stringify({
"model": "meta-llama/llama-3.1-8b-instruct",
"messages": [
{"role": "user", "content": "Hello"},
],
"provider": {
"quantizations": [
"fp8"
]
},
})
});
JSON Schema for Provider Preferences
For a complete list of options, see this JSON schema:
{
"$ref": "#/definitions/ProviderPreferences",
"definitions": {
"ProviderPreferences": {
"type": "object",
"properties": {
"allow_fallbacks": {
"type": [
"boolean",
"null"
],
"description": "Whether to allow backup providers to serve requests\n- true: (default) when the primary provider (or your custom providers in \"order\") is unavailable, use the next best provider.\n- false: use only the primary/custom provider, and return the upstream error if it's unavailable.\n"
},
"require_parameters": {
"type": [
"boolean",
"null"
],
"description": "Whether to filter providers to only those that support the parameters you've provided. If this setting is omitted or set to false, then providers will receive only the parameters they support, and ignore the rest."
},
"data_collection": {
"anyOf": [
{
"type": "string",
"enum": [
"deny",
"allow"
]
},
{
"type": "null"
}
],
"description": "Data collection setting. If no available model provider meets the requirement, your request will return an error.\n- allow: (default) allow providers which store user data non-transiently and may train on it\n- deny: use only providers which do not collect user data.\n"
},
"order": {
"anyOf": [
{
"type": "array",
"items": {
"type": "string",
"enum": [
"OpenAI",
"Anthropic",
"HuggingFace",
"Together",
"DeepInfra",
"Azure",
"Modal",
"AnyScale",
"Replicate",
"Perplexity",
"Recursal",
"Fireworks",
"Mistral",
"Groq",
"Cohere",
"Lepton",
"OctoAI",
"Novita",
"DeepSeek",
"Infermatic",
"AI21",
"Featherless",
"Lambda",
"Avian",
"SambaNova",
"01.AI",
"Google",
"Google AI Studio",
"Mancer",
"Mancer 2",
"Hyperbolic",
"Hyperbolic 2",
"Lynn 2",
"Lynn",
"Reflection"
]
}
},
{
"type": "null"
}
],
"description": "An ordered list of provider names. The router will attempt to use the first provider in the subset of this list that supports your requested model, and fall back to the next if it is unavailable. If no providers are available, the request will fail with an error message."
},
"ignore": {
"anyOf": [
{
"type": "array",
"items": {
"type": "string",
"enum": [
"OpenAI",
"Anthropic",
"HuggingFace",
"Together",
"DeepInfra",
"Azure",
"Modal",
"AnyScale",
"Replicate",
"Perplexity",
"Recursal",
"Fireworks",
"Mistral",
"Groq",
"Cohere",
"Lepton",
"OctoAI",
"Novita",
"DeepSeek",
"Infermatic",
"AI21",
"Featherless",
"Lambda",
"Avian",
"SambaNova",
"01.AI",
"Google",
"Google AI Studio",
"Mancer",
"Mancer 2",
"Hyperbolic",
"Hyperbolic 2",
"Lynn 2",
"Lynn",
"Reflection"
]
}
},
{
"type": "null"
}
],
"description": "List of provider names to ignore. If provided, this list is merged with your account-wide ignored provider settings for this request."
},
"quantizations": {
"anyOf": [
{
"type": "array",
"items": {
"type": "string",
"enum": [
"int4",
"int8",
"fp8",
"fp16",
"bf16",
"unknown"
]
}
},
{
"type": "null"
}
],
"description": "A list of quantization levels to filter the provider by."
}
},
"additionalProperties": false
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}