Embeddings: What They Cost and How to Not Overpay

Why embeddings rarely show up as a cost problem — until they do

Embedding models are priced per input token only — there's no output token cost, since the response is a fixed-length vector, not generated text. This makes embeddings dramatically cheaper per call than chat completions. The catch is volume: a RAG pipeline that re-embeds its entire document set on every deployment, rather than only the documents that changed, can rack up unnecessary cost purely through repetition, not through the per-token price being high.

What embedding models cost

| Model | Price (₹/1M tokens) | Provider | Notes | |-------|------------------------|----------|-------| | text-embedding-3-small | ~₹1.83 | OpenAI | Best default for most applications | | text-embedding-3-large | ~₹11.88 | OpenAI | ~6x the cost — use only if quality gain is measured | | text-embedding-ada-002 | ~₹9.14 | OpenAI | Legacy model, generally superseded by 3-small |

from openai import OpenAI
 
client = OpenAI(base_url="https://api.aicredits.in/v1", api_key="sk-your-aicredits-key")
 
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="AICredits is a unified LLM gateway with INR billing for Indian developers.",
)
vector = response.data[0].embedding  # 1536-dimensional vector

The real cost driver: re-embedding waste

A 10,000-document knowledge base at ~500 tokens/document is 5 million tokens — roughly ₹9.15 to embed once on text-embedding-3-small. That's cheap. The problem is teams that re-embed the entire set on every content update, every deploy, or every schema change, instead of only the documents that actually changed.

# Wasteful: re-embeds everything on every run
for doc in all_documents:
    embed_and_store(doc)
 
# Better: only embed documents that changed since last run
import hashlib
 
def content_hash(text: str) -> str:
    return hashlib.sha256(text.encode()).hexdigest()
 
for doc in all_documents:
    new_hash = content_hash(doc.content)
    if doc.id not in stored_hashes or stored_hashes[doc.id] != new_hash:
        embed_and_store(doc)
        stored_hashes[doc.id] = new_hash

At scale — a knowledge base with 100,000 documents re-indexed weekly instead of incrementally — this difference compounds into a real, avoidable cost over months, even though embeddings are individually cheap.

Dimension size vs. cost trade-off

text-embedding-3-small and text-embedding-3-large both support a dimensions parameter to reduce output vector size, which reduces storage and retrieval compute cost downstream — though not the embedding API cost itself, since that's priced on input tokens, not output dimensions:

response = client.embeddings.create(
    model="text-embedding-3-large",
    input="Your document text here.",
    dimensions=1024,  # smaller than the default 3072, cheaper to store and search
)

This matters more for vector database cost and query latency than for the embedding API bill directly, but it's worth knowing the two are tunable independently.

Frequently Asked Questions

Should I use text-embedding-3-small or text-embedding-3-large by default?

Start with text-embedding-3-small. It's roughly 6x cheaper and performs well for most retrieval tasks. Move to text-embedding-3-large only if you measure a retrieval quality improvement on your actual data, not by assumption.

Do I need to re-embed my documents if I switch embedding models?

Yes — vectors from different models (or even different dimension settings of the same model) aren't comparable, so switching requires a full re-embed of your document set.

Can I batch multiple texts in one embeddings request?

Yes — the input field accepts an array of strings, and you're billed for the total token count across all of them in a single request, which is more efficient than one request per document.

Get started

• Create a free account and test embeddings against your own documents.
• Related: LlamaIndex in Production: RAG Pipeline with Cost Tracking in ₹ and RAG Explained: Build an AI That Knows Your Own Data.