Prompt Caching Explained: Up to 90% Off Repeated Context

The waste hiding in your system prompt

If you send a 2,000-token system prompt with every single request — a coding assistant with the full file context, a support bot with your product catalog, a document-review tool with the same legal template — you are paying full input price to reprocess that same text every time, even though nothing in it changed.

Prompt caching fixes this. The provider caches the processed representation of your repeated context on the first request, then charges a steep discount for every subsequent request that hits the same cache.

How the pricing actually works

First request  → Cache WRITE → billed at 1.25× standard input rate
Second request → Cache HIT   → billed at 0.10× standard input rate (90% off)

You break even on the second request that reuses the same cached context. From the third request onward, you are pocketing the full discount. For a chatbot or coding assistant that fires the same system prompt hundreds of times a day, this compounds fast.

Enable it with a single flag through AICredits — no provider-specific SDK changes:

from openai import OpenAI
 
client = OpenAI(base_url="https://api.aicredits.in/v1", api_key="sk-your-key")
 
LARGE_SYSTEM_PROMPT = """You are an expert reviewer for [Company]'s legal contracts.
[... 2,000 tokens of instructions and reference clauses ...]"""
 
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.6",
    extra_body={"cache": True},
    messages=[
        {"role": "system", "content": LARGE_SYSTEM_PROMPT},
        {"role": "user", "content": "Review this NDA clause for unusual liability terms."},
    ],
)

The rupee math

Take a 2,000-token system prompt reused across 50 requests in a day, on a model priced at ₹274 per million input tokens (roughly Claude 3.5 Sonnet's rate through AICredits):

| Approach | Cost for the system-prompt portion (50 requests) | |----------|----------------------------------------------------| | No caching | 50 × 2,000 tokens × ₹274/1M = ₹27.40 | | With caching | 1 write (₹0.685) + 49 hits (49 × 2,000 × ₹27.40/1M = ₹2.68) = ₹3.37 |

That's roughly an 88% reduction on the repeated-context portion of the bill — and the saving scales linearly with how often the same context gets reused.

Provider support and limits

| Provider | Caching method | Notes | |----------|-----------------|-------| | Anthropic (Claude) | Explicit — set "cache": true | Cache TTL is 5 minutes, refreshed on every hit | | OpenAI (GPT-4o family) | Automatic | Applies for prompts over a minimum length — no flag needed | | Other providers | Not supported | Caching is currently Claude and OpenAI only |

The minimum cacheable size is roughly 1,000 tokens (~4,000 characters) — caching a short system prompt won't hit the threshold and provides no benefit. The cache key is derived from the start of your system message, so template the variable parts to come after the static content, not interleaved with it. Even a single-character change to the cached prefix invalidates the cache and forces a fresh write.

When caching costs you money instead of saving it

If your system prompt is only ever sent once — a one-off script, a low-traffic endpoint, a prompt that changes on every call — caching adds the 1.25× write premium with no hits to offset it. Only enable it for context you know will be reused at least twice within the 5-minute TTL window.

Frequently Asked Questions

Does prompt caching change my code beyond the flag?

No. Message format, streaming, and response structure are unchanged. For Claude models, add "cache": true to the request body; for OpenAI models, caching applies automatically once the prompt exceeds the minimum length.

What happens if the cache expires between requests?

The next request after the 5-minute TTL expires is billed as a fresh cache write (1.25×), not a hit. If your traffic pattern has long gaps between requests, caching may not help — the discount only applies within the TTL window.

Can I cache the conversation history, not just the system prompt?

The cache is keyed on the start of your message sequence, typically the system prompt and any static leading content. Rapidly changing conversation history further down the message list is not cached — only the stable prefix benefits.

Get started

• Read the full prompt caching docs for the exact API reference.
• Create a free account and test caching against your own system prompt.
• Related: Semantic Caching: Cut LLM API Costs by 40% on Repeated Queries — a different technique for identical or near-identical user queries.