Prompt Caching Explained: The 90% Discount Most Teams Never Claim (2026)

Every turn of an AI agent resends the same thing: the system prompt, every tool schema, the whole conversation so far. By turn forty, that's a hundred thousand tokens the model has already read thirty-nine times — and without prompt caching, you pay full price to have it read them a fortieth.

Prompt caching is an LLM API feature that stores the processed beginning of a prompt on the provider's side, so repeated requests with the same prefix skip that work. Cached tokens cost about a tenth of the normal input price on Anthropic, OpenAI and Gemini — the single biggest cost lever for AI agents, bigger than model choice.

The reason most teams never collect: the failure is silent. Nothing errors, nothing warns. The response is identical whether the cache hit or missed. Only the invoice knows.

What Is Prompt Caching? The 101

Think of a tutor who re-reads the entire textbook before answering each question. Prompt caching is the bookmark: the provider keeps the tutor's place, so the next question starts from page 400 instead of page 1.

Under the hood, every request goes through a prefill step: the model reads your whole prompt and computes attention state for each token before it writes a single word of output. That state depends only on the tokens that came before it. So if two requests share the same first 50,000 tokens, the state for those tokens is identical — and the provider can compute it once, store it, and reuse it.

That's the entire trick. Three consequences follow from it, and every rule in this article is one of them:

• It's prefix-only. The cache covers the prompt from the start up to the last matching token. Nothing after the first difference is reusable.
• It's exact-match. One changed byte at position 500 invalidates everything from 500 onward. The cache doesn't patch itself around a small edit.
• It expires. Entries live minutes, not days. Come back after lunch and you may start cold.

Prompt caching does not change the answer. The model still reads the full context and still generates fresh output; it just skips recomputing state it already computed. It is not the same as response caching (returning a stored answer to a repeated question) or semantic caching (returning a stored answer to a similar one).

Why AI Agents Are the Perfect Customer

For a chatbot answering one-off questions, caching is a nice-to-have. The prompt is short and rarely repeats.

An agent is the opposite. Its prompt is system instructions + tool definitions + every prior turn, and every turn appends to it. Tool results come back in as input, so input tokens dwarf output tokens. Most of what the model reads on turn 40 is byte-identical to what it read on turn 39.

Here's what that means in dollars. Take a $3-per-million-token input price and a 40-turn session: a 20,000-token stable prefix (system prompt plus tool schemas) that grows by 2,000 tokens per turn, with every turn landing within five minutes of the last.

Same model. Same session. 85% less. Every token is written to the cache exactly once — at a 25% premium — and read from it on every turn after that, at 90% off. The longer the session, the closer the saving gets to 90%, because reads dominate.

That's why teams that never turned this on end up with spiraling tokenmaxxing bills — and why context engineering treats cache behavior as a first-class design input, not a billing footnote.

Prompt Caching by Provider (2026)

All three major providers now offer it, and all three read cached tokens at roughly a tenth of the base price. The mechanics differ enough to bite.

Sources: Anthropic, OpenAI, Gemini docs, checked September 2026.

Two things stand out. First, "automatic" doesn't mean "working": OpenAI and Gemini cache on their own, but only if your prefix is actually stable — and the same invalidators apply. Second, Anthropic makes you place markers, but in exchange it tells you exactly what got written and what got read. That visibility is how you find leaks.

How Anthropic's cache_control Works

Anthropic's version is the most explicit, so it's the best one to learn the mechanics on.

The prompt renders in a fixed order: tools, then system, then messages. A cache breakpoint on the last system block caches the tools and the system prompt together. That's the basic pattern:

{
	"model": "claude-opus-5",
	"max_tokens": 16000,
	"system": [
		{
			"type": "text",
			"text": "<your long, stable system prompt>",
			"cache_control": { "type": "ephemeral" }
		}
	],
	"messages": [{ "role": "user", "content": "..." }]
}

Or skip the bookkeeping: put a single cache_control field at the top level of the request, and the API places the breakpoint on the last cacheable block and moves it forward as the conversation grows. For a multi-turn agent, that's usually the right default — though it still pays to keep one explicit marker on the system prompt, so the expensive shared part always has a guaranteed read point.

The rules that matter:

• Up to 4 breakpoints per request. Spend them at stability boundaries: tools never change, system changes per deploy, conversation changes per turn.
• Minimum cacheable length is per-model, and not monotonic. 512 tokens on Claude Opus 5 and Fable 5, 1,024 on Sonnet 5 and Opus 4.8, 2,048 on Opus 4.7, 4,096 on Opus 4.6 and Haiku 4.5. Below the minimum, the marker is silently ignored — no error, just cache_creation_input_tokens: 0.
• Writes cost 1.25× base input for the 5-minute TTL, 2× for the 1-hour TTL. Reads cost 0.1×. With the 5-minute cache, the second request already pays for the first: 1.25 + 0.1 = 1.35 versus 2.0 uncached. The 1-hour cache needs three requests to break even, so it's only worth it when gaps between requests run 5–60 minutes.
• Every hit refreshes the timer at no cost. Requests less than five minutes apart keep the default cache warm indefinitely. The lifetime counts from the start of the request, so a four-minute generation leaves one minute of slack.
• Cache hits don't count against your rate limits. A warm cache raises effective throughput, not just savings.
• The lookback window is 20 blocks. Each breakpoint searches at most 20 positions backward for a prior cache entry. A single turn that appends more than 20 content blocks (long sequential tool loops) can push the previous entry out of range and silently miss. Runs of consecutive tool_use or tool_result blocks count as one position, so parallel tool calls are safe.
• Parallel requests don't share a cache that isn't written yet. An entry becomes readable only after the first response begins. Fire ten identical requests at once and all ten pay full price. Send one, wait for the first token, then send the other nine.

The Catch: One Byte Breaks It

Here's where the money leaks. A cache is only as good as its prefix, and anything that changes the prefix invalidates the cache from that point on.

Anthropic documents the hierarchy precisely. Changes at each level invalidate that level and everything after it:

Two rows deserve a second look. Tool definitions sit at position zero, so adding one tool mid-session — a common "load on demand" pattern — rebuilds everything. And caches are model-scoped: routing a cheap sub-task to a smaller model on the same conversation gets zero reuse.

The invalidators that actually bite in production are rarely deliberate. They look like this:

The fix is the same every time: move the dynamic piece after the last breakpoint, make it deterministic, or delete it if it isn't load-bearing.

This is also why "just summarize the context every N turns" backfires so often. A summary rewrites the middle of your prefix. You save tokens on the next input and pay full price to rebuild the cache — which brings us to the math.

Compression Can Cost More Than It Saves

Take a session sitting at 100,000 cached tokens, on a $3-per-million model. Each turn currently reads those tokens at $0.30 per million: 3 cents a turn.

Now compress it to 30,000 tokens:

• The summarizer reads 100,000 tokens at full price: $0.30, plus its output.
• The new 30,000-token context is written to cache at 1.25×: $0.11.
• Each subsequent turn reads 30,000 tokens: 0.9 cents. You save 2.1 cents a turn.

Break-even is roughly 20 turns later. If the session ends in ten, compression lost money — and you also lost whatever detail the summary dropped. The "optimization" was the leak. This is the trap the context rot fix walks into when it's applied on a timer instead of on the numbers.

The principle generalizes: the cheapest context is often the biggest one, because it's already cached. Rewriting is the expensive operation, not carrying.

How to Tell If Your Cache Is Actually Working

Don't trust the docs, and don't trust your code. Trust the usage fields on every response. On Anthropic there are three, and the total prompt is their sum:

• cache_read_input_tokens — served from cache at 0.1×
• cache_creation_input_tokens — written this request at 1.25× (or 2×)
• input_tokens — everything after the last breakpoint, at full price

A healthy agent loop has an unmistakable signature: reads grow every turn and cover the whole prior conversation; writes are small — just what the last turn added; input_tokens is a sliver. Two red flags:

1. Reads are zero across repeated requests. A silent invalidator is upstream of your first breakpoint. Diff the rendered bytes of two consecutive requests and you'll find it.
2. Writes are near the full conversation size on every turn. Something rewrites the prefix each time — a compression step, a mutating history, a tool list that isn't deterministic — or a turn appended more than 20 blocks and fell out of the lookback window.

And the costliest failure is a regression, not a bad first setup. Caching worked when it was written; then a later change added a dynamic field to the system prompt, and for months every request missed while everything kept succeeding. Put a check on it: an integration test that asserts a second identical request shows cache_read_input_tokens > 0 catches this in CI instead of in the invoice.

How Octomind Spends Its Four Breakpoints

Octomind is built around this arithmetic, and the difference shows up in how it spends the four markers Anthropic allows.

The markers. The system prompt carries a marker with the 1-hour TTL, and the last tool definition carries another — also 1-hour — so the tool block and the system block are cached together and survive a coffee break. The remaining two markers move along the conversation. When Octomind compresses, the summary block becomes the new stable cache boundary, so the fold itself doesn't strand the prefix.

Compression does the math before it acts. Octomind's compression engine is cache-aware. Before compressing, it weighs two numbers: what it costs to carry the full context through the calls the session is expected to make — paced from the session's own history — and what it costs to compress now, meaning the summarizer's input and output plus re-writing the smaller context at the cache-write price. If the fold wouldn't be amortized over the predicted work, it doesn't fire. Sometimes the cheapest move is carrying 200k tokens, because they're 90%-off tokens and rewriting them is full price.

The cache can be kept warm while you think. When you walk away after a reply, the TTL counts down and the next turn may start cold. Cache keepalive — an opt-in setting — sends minimal max_tokens = 1 pings against a frozen snapshot of the conversation at the interval the provider recommends:

cache_keepalive_enabled = true           # default: false
cache_keepalive_max_idle_seconds = 1800  # stop pinging 30 min after last activity

Each ping costs a cache read, which is the point — a cache read is the cheapest thing you can buy.

Tools load on demand and stay loaded. Capabilities activate when a task needs them and stay resident under an LRU cap, so the tool block — the front of your prefix — changes when you switch domains, not every turn. That's a deliberate one-time rebuild instead of a permanent tax of forty schemas on every request.

You can watch all of it. /info inside a session breaks down input, output, cached and reasoning tokens, estimated cache savings, and which markers are placed on what. If the cached share of your input is low on a long session, something is churning your prefix — and now you know where to look. The cost-cutting guide walks through reading it.

The Prompt Caching Checklist

Running any agent against an API? Six checks, in order.

1. Is caching on, and is it hitting? Anthropic needs cache_control markers or the top-level auto field; OpenAI and Gemini are automatic. Confirm in the usage fields, not in the docs.
2. Is your prefix above the minimum? 512 to 4,096 tokens on Anthropic depending on model, 1,024 on OpenAI, 2,048 or 4,096 on Gemini. Below that, nothing caches and nothing tells you.
3. Is your prefix stable? Stable content first — tool definitions, system prompt — with anything dynamic (timestamps, IDs, per-request context) after the last breakpoint. Serialize deterministically.
4. Are you switching tools or models mid-session? Both rebuild the cache from zero. Load tools once per task; keep one model per conversation and spawn a separate sub-agent for cheap work.
5. Are you compressing more than you should? Run the numbers with cache invalidation included. If the session is likely to end before break-even, don't.
6. Are you measuring per session? Cached versus uncached input share is the one metric that tells you whether the discount you're owed is the discount you're getting.

The model is the expensive part, but the prefix is the part you control. Ninety percent off is sitting there in every major provider's API. Claim it.

Get Octomind — and run /info on your next session. The cached line will surprise you.

FAQ

What is prompt caching? Prompt caching is a provider-side feature that stores the processed state of the beginning of your prompt — the system prompt, tool definitions and prior conversation — so repeated requests with the same prefix don't reprocess those tokens. Cached reads cost about a tenth of the base input price on Anthropic, OpenAI (GPT-5.6 and later) and Gemini. For agents that resend the same context every turn, it's the largest single cost lever available.

How much does prompt caching save? Cached tokens are billed at 0.1× the base input rate on Anthropic, on OpenAI's GPT-5.6 and newer, and on Gemini. Anthropic charges a write premium of 1.25× (5-minute cache) or 2× (1-hour cache), which pays for itself by the second or third request. On long agent sessions where most of each prompt is repeated prefix, total input cost typically drops 80–90%; the worked 40-turn example above goes from $7.32 to $1.08.

Does prompt caching change the model's answers? No. The model still reads the entire context and generates fresh output every request. Caching only skips recomputing the internal state for a prefix it has already processed. It is different from response caching or semantic caching, which return a stored answer instead of generating one.

Does prompt caching work with OpenAI models? Yes, and it's automatic on supported models. On GPT-5.6 and later, cached input tokens bill at 0.1× the uncached rate, the minimum prefix is 1,024 tokens, and a cached prefix stays reusable for 30 minutes after its last use. Older models bill a model-specific cached rate and cache for 5–10 minutes by default, with an optional 24-hour retention. Check usage.input_tokens_details.cached_tokens to see hits.

Does Gemini have prompt caching? Yes. Gemini has had implicit caching on by default for Gemini 2.5 and newer since May 2025, with cached tokens billed at a tenth of the input price (Gemini 2.5 Flash: $0.03 versus $0.30 per million). Minimum prefix is 2,048 tokens on 2.5 models and 4,096 on newer ones. Explicit caches let you pin content for a chosen lifetime, at a per-hour storage cost.

Why is my cache hit rate zero? Almost always one of four things: your prefix is below the model's minimum length; something dynamic (a timestamp, a request ID, an unsorted JSON object) sits before your first breakpoint; your tool list or model changes between requests; or requests arrive more than the TTL apart. Diff the rendered bytes of two consecutive requests — the first differing byte is your leak.

Does prompt caching reduce latency too? Yes. Prefill — reading and processing the input before the first output token — is what caching skips, so time-to-first-token drops along with cost. On Anthropic, cache hits also don't count toward input-token rate limits, so a warm cache raises throughput as well.

Does compressing context help or hurt caching? Both, depending on the numbers. Compression shrinks future reads but invalidates the cache from the rewrite point and costs a full-price summarization pass. It pays off only when the session runs long enough after the fold to amortize that — often 20 or more turns. Octomind's compression engine computes this before each fold and skips folds that would lose money.