Rate Limits, Budgets, and Per-Key Controls: Taming Team AI Spend

The problem with account-level spending limits

Setting a single spending cap on your OpenAI or Anthropic account protects the account as a whole, but it doesn't tell you which key, which environment, or which team member caused a spike. When the cap trips, every integration using that account stops at once — your production app alongside a developer's test script.

AICredits puts budget and rate controls at the API key level instead, so each key — production, staging, a specific team, a specific developer — has its own independent limits drawing from the shared wallet.

Setting a per-key budget

In the dashboard, under API Keys → Create/Edit Key, set a Budget (₹). Once a key's cumulative usage crosses that number, further requests on that key return a 402 — other keys on the account are unaffected.

Account wallet: ₹10,000
  ├─ key: prod-backend       (no budget cap — trusted, monitored via alerts)
  ├─ key: staging            (budget: ₹500)
  ├─ key: dev-alice          (budget: ₹200)
  └─ key: dev-bob            (budget: ₹200)

This structure means a bug in a developer's local script can burn through at most ₹200, never touching the production key's headroom.

Setting RPM and concurrency limits

Each key also carries a requests-per-minute limit (default 60, configurable up to 10,000) and a concurrency limit (default 5 simultaneous requests). Both matter for different failure modes:

• RPM limit protects against a loop that fires requests faster than intended — a retry without backoff, a webhook handler triggered repeatedly.
• Concurrency limit protects against a burst of simultaneous requests overwhelming downstream systems, even if the total-per-minute count looks fine.

# Every response includes these headers so you can monitor usage in real time
HTTP/1.1 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 35

A practical setup for a small team

| Key purpose | Budget | RPM | Concurrency | |-------------|--------|-----|--------------| | Production backend | ₹8,000/month, monitored | 500 | 20 | | Staging environment | ₹500/month | 60 | 5 | | CI/automated tests | ₹100/month | 30 | 3 | | Individual developer sandboxes | ₹200/month each | 30 | 3 |

Set the CI key's budget deliberately low — automated test suites that call a real LLM on every run can rack up cost quickly if a test loops or retries unexpectedly, and a tight cap catches this before it becomes a real bill.

Handling 429s gracefully once limits are in place

import time, random
from openai import OpenAI, RateLimitError
 
client = OpenAI(base_url="https://api.aicredits.in/v1", api_key="sk-your-key")
 
def chat_with_retry(messages, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(model="openai/gpt-4o-mini", messages=messages)
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)

A 429 from a budget cap (402, technically — distinct from rate-limit 429s) means the key is out of allocated spend, not that the request should be retried — check the status code to distinguish "slow down" from "budget exhausted" and handle each differently in your application logic.

Frequently Asked Questions

Can I raise a key's budget after it's been created?

Yes — edit the key in the dashboard and increase the budget figure. The change takes effect immediately for subsequent requests.

What happens to in-flight requests when a budget cap is hit mid-request?

Requests already accepted before the cap was reached complete normally. Only new requests submitted after the cap is exceeded are rejected.

Is there a way to get alerted before a key hits its budget, not just after?

Check your account's low-balance alert settings — AICredits surfaces a low-balance warning at the wallet level; for per-key budget proximity, monitor usage via the dashboard or your own logging of the X-RateLimit-* headers and periodic balance checks.

Do concurrency limits apply per key or per account?

Concurrency limits are enforced per user account by default, capping total simultaneous active requests across all your keys — check your plan's specific limit in the dashboard.

Get started

• Read the rate limits docs for the full reference.
• Create a free account and set up separate keys for your environments.
• Related: Agentic AI Costs: How One Loop Burned ₹5,000 in 10 Minutes.