Provider Routing
Route requests to the best provider
OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers to maximize uptime.
You can customize how your requests are routed using the provider
object in the request body for Chat Completions and Completions.
For a complete list of valid provider names to use in the API, see the full provider schema.
The provider
object can contain the following fields:
Price-Based Load Balancing (Default Strategy)
For each model in your request, OpenRouter’s default behavior is to load balance requests across providers, prioritizing price.
If you are more sensitive to throughput than price, you can use the sort
field to explicitly prioritize throughput.
When you send a request with tools
or tool_choice
, OpenRouter will only
route to providers that support tool use. Similarly, if you set a
max_tokens
, then OpenRouter will only route to providers that support a
response of that length.
Here is OpenRouter’s default load balancing strategy:
- Prioritize providers that have not seen significant outages in the last 30 seconds.
- For the stable providers, look at the lowest-cost candidates and select one weighted by inverse square of the price (example below).
- Use the remaining providers as fallbacks.
A Load Balancing Example
If Provider A costs $1 per million tokens, Provider B costs $2, and Provider C costs $3, and Provider B recently saw a few outages.
- Your request is routed to Provider A. Provider A is 9x more likely to be first routed to Provider A than Provider C because (inverse square of the price).
- If Provider A fails, then Provider C will be tried next.
- If Provider C also fails, Provider B will be tried last.
If you have sort
or order
set in your provider preferences, load balancing will be disabled.
Provider Sorting
As described above, OpenRouter load balances based on price, while taking uptime into account.
If you instead want to explicitly prioritize a particular provider attribute, you can include the sort
field in the provider
preferences. Load balancing will be disabled, and the router will try providers in order.
The three sort options are:
"price"
: prioritize lowest price"throughput"
: prioritize highest throughput"latency"
: prioritize lowest latency
To always prioritize low prices, and not apply any load balancing, set sort
to "price"
.
To always prioritize low latency, and not apply any load balancing, set sort
to "latency"
.
Nitro Shortcut
You can append :nitro
to any model slug as a shortcut to sort by throughput. This is exactly equivalent to setting provider.sort
to "throughput"
.
Floor Price Shortcut
You can append :floor
to any model slug as a shortcut to sort by price. This is exactly equivalent to setting provider.sort
to "price"
.
Ordering Specific Providers
You can set the providers that OpenRouter will prioritize for your request using the order
field.
The router will prioritize providers in this list, and in this order, for the model you’re using. If you don’t set this field, the router will load balance across the top providers to maximize uptime.
OpenRouter will try them one at a time and proceed to other providers if none are operational. If you don’t want to allow any other providers, you should disable fallbacks as well.
Example: Specifying providers with fallbacks
This example skips over OpenAI (which doesn’t host Mixtral), tries Together, and then falls back to the normal list of providers on OpenRouter:
Example: Specifying providers with fallbacks disabled
Here’s an example with allow_fallbacks
set to false
that skips over OpenAI (which doesn’t host Mixtral), tries Together, and then fails if Together fails:
Requiring Providers to Support All Parameters (beta)
You can restrict requests only to providers that support all parameters in your request using the require_parameters
field.
With the default routing strategy, providers that don’t support all the LLM parameters specified in your request can still receive the request, but will ignore unknown parameters. When you set require_parameters
to true
, the request won’t even be routed to that provider.
Example: Excluding providers that don’t support JSON formatting
For example, to only use providers that support JSON formatting:
Requiring Providers to Comply with Data Policies
You can restrict requests only to providers that comply with your data policies using the data_collection
field.
allow
: (default) allow providers which store user data non-transiently and may train on itdeny
: use only providers which do not collect user data
Some model providers may log prompts, so we display them with a Data Policy tag on model pages. This is not a definitive source of third party data policies, but represents our best knowledge.
Account-Wide Data Policy Filtering
This is also available as an account-wide setting in your privacy settings. You can disable third party model providers that store inputs for training.
Example: Excluding providers that don’t comply with data policies
To exclude providers that don’t comply with your data policies, set data_collection
to deny
:
Disabling Fallbacks
To guarantee that your request is only served by the top (lowest-cost) provider, you can disable fallbacks.
This is combined with the order
field from Ordering Specific Providers to restrict the providers that OpenRouter will prioritize to just your chosen list.
Ignoring Providers
You can ignore providers for a request by setting the ignore
field in the provider
object.
Ignoring multiple providers may significantly reduce fallback options and limit request recovery.
Account-Wide Ignored Providers
You can ignore providers for all account requests by configuring your preferences. This configuration applies to all API requests and chatroom messages.
Note that when you ignore providers for a specific request, the list of ignored providers is merged with your account-wide ignored providers.
Example: Ignoring Azure for a request calling GPT-4 Omni
Here’s an example that will ignore Azure for a request calling GPT-4 Omni:
Quantization
Quantization reduces model size and computational requirements while aiming to preserve performance. Most LLMs today use FP16 or BF16 for training and inference, cutting memory requirements in half compared to FP32. Some optimizations use FP8 or quantization to reduce size further (e.g., INT8, INT4).
Quantized models may exhibit degraded performance for certain prompts, depending on the method used.
Providers can support various quantization levels for open-weight models.
Quantization Levels
By default, requests are load-balanced across all available providers, ordered by price. To filter providers by quantization level, specify the quantizations
field in the provider
parameter with the following values:
int4
: Integer (4 bit)int8
: Integer (8 bit)fp4
: Floating point (4 bit)fp6
: Floating point (6 bit)fp8
: Floating point (8 bit)fp16
: Floating point (16 bit)bf16
: Brain floating point (16 bit)fp32
: Floating point (32 bit)unknown
: Unknown
Example: Requesting FP8 Quantization
Here’s an example that will only use providers that support FP8 quantization:
Terms of Service
You can view the terms of service for each provider below. You may not violate the terms of service or policies of third-party providers that power the models on OpenRouter.
OpenAI
: https://openai.com/policies/row-terms-of-use/Anthropic
: https://www.anthropic.com/legal/commercial-termsGoogle Vertex
: https://cloud.google.com/terms/Google AI Studio
: https://cloud.google.com/terms/Amazon Bedrock
: https://aws.amazon.com/service-terms/Groq
: https://groq.com/terms-of-use/SambaNova
: https://sambanova.ai/terms-and-conditionsCohere
: https://cohere.com/terms-of-useMistral
: https://mistral.ai/terms/#terms-of-useTogether
: https://www.together.ai/terms-of-serviceTogether (lite)
: https://www.together.ai/terms-of-serviceFireworks
: https://fireworks.ai/terms-of-serviceDeepInfra
: https://deepinfra.com/docs/dataLepton
: https://www.lepton.ai/policies/tosNovitaAI
: https://novita.ai/legal/terms-of-serviceAvian.io
: https://avian.io/privacyLambda
: https://lambdalabs.com/legal/privacy-policyAzure
: https://www.microsoft.com/en-us/legal/terms-of-use?oneroute=trueModal
: https://modal.com/legal/termsAnyScale
: https://www.anyscale.com/termsReplicate
: https://replicate.com/termsPerplexity
: https://www.perplexity.ai/hub/legal/perplexity-api-terms-of-serviceRecursal
: https://featherless.ai/termsOctoAI
: https://octo.ai/docs/faqs/privacy-and-securityDeepSeek
: https://chat.deepseek.com/downloads/DeepSeek%20Terms%20of%20Use.htmlInfermatic
: https://infermatic.ai/privacy-policy/AI21
: https://studio.ai21.com/privacy-policyFeatherless
: https://featherless.ai/termsInflection
: https://developers.inflection.ai/tosxAI
: https://x.ai/legal/terms-of-serviceCloudflare
: https://www.cloudflare.com/service-specific-terms-developer-platform/#developer-platform-termsSF Compute
: https://inference.sfcompute.com/privacyMinimax
: https://intl.minimaxi.com/protocol/terms-of-serviceNineteen
: https://nineteen.ai/tosLiquid
: https://www.liquid.ai/terms-conditionsinference.net
: https://inference.net/termsFriendli
: https://friendli.ai/terms-of-serviceAionLabs
: https://www.aionlabs.ai/terms/Alibaba
: https://www.alibabacloud.com/help/en/legal/latest/alibaba-cloud-international-website-product-terms-of-service-v-3-8-0Nebius AI Studio
: https://docs.nebius.com/legal/studio/terms-of-use/Chutes
: https://chutes.ai/toskluster.ai
: https://www.kluster.ai/terms-of-useCrusoe
: https://legal.crusoe.ai/open-router#managed-inference-tos-open-routerTargon
: https://targon.com/termsUbicloud
: https://www.ubicloud.com/docs/about/terms-of-serviceParasail
: https://www.parasail.io/legal/terms01.AI
: https://platform.01.ai/privacypolicyHuggingFace
: https://huggingface.co/terms-of-serviceMancer
: https://mancer.tech/termsMancer (private)
: https://mancer.tech/termsHyperbolic
: https://hyperbolic.xyz/privacyHyperbolic (quantized)
: https://hyperbolic.xyz/privacyLynn
: https://api.lynn.app/policy
JSON Schema for Provider Preferences
For a complete list of options, see this JSON schema: