Provider Routing
How AICredits routes requests across LLM providers. Primary provider selection, secondary routes, per-request overrides, and automatic failover.
Use this page with an AI assistant
Opens a new chat with this docs URL and the correct AICredits base URLs.
AICredits routes every request to the right upstream provider based on the model ID you specify. This page explains the routing logic and how to control it.
How a Route Is Chosen
Every request resolves your model to a primary provider plus an ordered list of secondary routes. The gateway tries them in order until one succeeds.
- Model ID. Use the
provider/model-nameform (for exampleopenai/gpt-4o-minioranthropic/claude-sonnet-4.6) to name the primary provider explicitly. A bare name such asgpt-4o-miniorclaude-sonnet-4.6is matched by its well-known prefix (gpt,o1/o3/o4,claude,gemini,deepseek,mistral,grok, and a few others). A model with an unrecognised prefix is served from the aggregated provider pool. - Configured routes. Many models are available from more than one provider. Where AICredits has configured several routes for a model, they are tried in priority order, lowest priority number first.
- Last-resort route. An aggregated provider pool is appended at the end of the chain so that a model-not-found or provider outage on a direct route can still be recovered.
Use explicit provider/model IDs in production. Prefix detection is a convenience for quick tests and can be ambiguous when providers name new models similarly.
Per-Request Overrides
AICredits extends the chat completions body with two optional routing controls.
provider: preferred order and fallback control
{
"model": "anthropic/claude-sonnet-4.6",
"messages": [{ "role": "user", "content": "Hello!" }],
"provider": {
"order": ["anthropic"],
"allow_fallbacks": false
}
}| Field | Behaviour |
|---|---|
provider.order | Providers to try, in the order given (for example ["anthropic", "bedrock", "vertex"]). Names are case-insensitive. A provider that cannot serve the requested model is skipped. |
provider.allow_fallbacks | Defaults to true. Set false to restrict the request to the providers in order; the aggregated last-resort route is not added. |
models: cross-model fallback
Send a models array to try several models in order. See Models & Providers. When models has more than one entry, each model uses its own normal route chain, and provider.order is not applied.
Failover
Within a chain, failures move the request to the next route:
- Healthy keys first. Provider keys are used round-robin, skipping keys that are currently marked unhealthy.
- Next route. When a route fails (for example a provider error, rate limit or model-not-found), the request continues on the next route in the chain.
A circuit breaker marks a provider key as unhealthy for 30 seconds after repeated failures, then recovers automatically.
Which provider served my request?
Every chat completion response includes an X-AICredits-Provider header with the provider that served it (for example openai, anthropic or google). Requests served by the aggregated pool report aggregated.
Supported Providers
| Provider | Example models |
|---|---|
| OpenAI | openai/gpt-4o-mini, openai/gpt-4o, openai/text-embedding-3-small, TTS and Whisper |
| Anthropic | anthropic/claude-sonnet-4.6, anthropic/claude-haiku-4.5 |
google/gemini-2.5-flash, google/gemini-2.5-pro | |
| DeepSeek | deepseek/deepseek-chat, deepseek/deepseek-v4-flash |
| Mistral, xAI, Groq, Together AI | Models listed on the Models page |
| Cloud platforms | Anthropic and other models can also be served from Amazon Bedrock and Google Vertex AI where configured |
| Sarvam | Indian-language TTS and STT |
| Aggregated pool | Additional models, and last-resort fallback |
Availability changes over time. The Models page and GET /api/models are the source of truth.
Model Discovery
To see all available models with their provider prefixes, use the models endpoint:
curl https://api.aicredits.in/api/modelsOr browse Models → for a searchable list with pricing.