Skip to main content
Features

Provider Routing

How AICredits routes requests across LLM providers. Primary provider selection, secondary routes, per-request overrides, and automatic failover.

Use this page with an AI assistant

Opens a new chat with this docs URL and the correct AICredits base URLs.

AICredits routes every request to the right upstream provider based on the model ID you specify. This page explains the routing logic and how to control it.

How a Route Is Chosen

Every request resolves your model to a primary provider plus an ordered list of secondary routes. The gateway tries them in order until one succeeds.

  1. Model ID. Use the provider/model-name form (for example openai/gpt-4o-mini or anthropic/claude-sonnet-4.6) to name the primary provider explicitly. A bare name such as gpt-4o-mini or claude-sonnet-4.6 is matched by its well-known prefix (gpt, o1/o3/o4, claude, gemini, deepseek, mistral, grok, and a few others). A model with an unrecognised prefix is served from the aggregated provider pool.
  2. Configured routes. Many models are available from more than one provider. Where AICredits has configured several routes for a model, they are tried in priority order, lowest priority number first.
  3. Last-resort route. An aggregated provider pool is appended at the end of the chain so that a model-not-found or provider outage on a direct route can still be recovered.

Use explicit provider/model IDs in production. Prefix detection is a convenience for quick tests and can be ambiguous when providers name new models similarly.

Per-Request Overrides

AICredits extends the chat completions body with two optional routing controls.

provider: preferred order and fallback control

Prefer one provider, no fallbacks
{
  "model": "anthropic/claude-sonnet-4.6",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "provider": {
    "order": ["anthropic"],
    "allow_fallbacks": false
  }
}
FieldBehaviour
provider.orderProviders to try, in the order given (for example ["anthropic", "bedrock", "vertex"]). Names are case-insensitive. A provider that cannot serve the requested model is skipped.
provider.allow_fallbacksDefaults to true. Set false to restrict the request to the providers in order; the aggregated last-resort route is not added.

models: cross-model fallback

Send a models array to try several models in order. See Models & Providers. When models has more than one entry, each model uses its own normal route chain, and provider.order is not applied.

Failover

Within a chain, failures move the request to the next route:

  1. Healthy keys first. Provider keys are used round-robin, skipping keys that are currently marked unhealthy.
  2. Next route. When a route fails (for example a provider error, rate limit or model-not-found), the request continues on the next route in the chain.

A circuit breaker marks a provider key as unhealthy for 30 seconds after repeated failures, then recovers automatically.

Which provider served my request?

Every chat completion response includes an X-AICredits-Provider header with the provider that served it (for example openai, anthropic or google). Requests served by the aggregated pool report aggregated.

Supported Providers

ProviderExample models
OpenAIopenai/gpt-4o-mini, openai/gpt-4o, openai/text-embedding-3-small, TTS and Whisper
Anthropicanthropic/claude-sonnet-4.6, anthropic/claude-haiku-4.5
Googlegoogle/gemini-2.5-flash, google/gemini-2.5-pro
DeepSeekdeepseek/deepseek-chat, deepseek/deepseek-v4-flash
Mistral, xAI, Groq, Together AIModels listed on the Models page
Cloud platformsAnthropic and other models can also be served from Amazon Bedrock and Google Vertex AI where configured
SarvamIndian-language TTS and STT
Aggregated poolAdditional models, and last-resort fallback

Availability changes over time. The Models page and GET /api/models are the source of truth.

Model Discovery

To see all available models with their provider prefixes, use the models endpoint:

curl https://api.aicredits.in/api/models

Or browse Models → for a searchable list with pricing.