Prompt caching for OpenAI, Claude and Gemini: prices, cache hits and a checklist for your backend

Prompt caching is the cheapest optimization most AI features never turn on properly. Every major LLM API can reuse the processed start of a prompt it has seen recently, and charge a fraction of the normal input price for it. If your backend sends the same instructions, tool definitions or document with every request, prompt caching can cut that part of the bill by 90% or more and shorten the time to the first token. If the prompt is built the wrong way, the cache never hits and you pay full price without noticing.

The rules also moved in the last few months. OpenAI now charges for cache writes on GPT-5.6 and later, Anthropic prices cache reads on Claude Opus 5.5 and Sonnet 5.5 at half its usual cache read rate, and Google's newest Gemini models need at least 4,096 tokens before anything is cached. This guide compares the three APIs as their documentation stands in October 2026, shows the break-even math, gives Node.js code for each, and lists the reasons a cache stays cold.

Key takeaways

• Prompt caching reuses an identical prefix. Anything that changes early in the prompt, even a timestamp, makes everything after it a cache miss.
• Prices differ. Claude charges 1.25 times the input price to write a 5-minute cache and 2 times for 1 hour; GPT-5.6 and later charge 1.25 times for writes. Reads cost 0.1 times the input price on most models, and 0.05 times on Claude Opus 5.5 and Sonnet 5.5.
• Minimum sizes apply. From 512 tokens on the newest Claude models to 4,096 tokens on Gemini 3.8 Flash. Shorter prompts are processed normally, with no error.
• With a write charge, a prefix used only once costs more than no caching. Two uses already pay off at 1.25 times write and 0.1 times read.
• Measure the hit rate from the usage fields on every response and alert when it drops, because a prompt change can silently turn caching off.

What prompt caching does#

An LLM processes every input token before it writes the first output token. When many requests start with the same tokens, the provider can keep the processed state of that prefix and skip the work next time. You pay a reduced rate for the reused tokens and the response starts sooner.

Three properties shape everything else:

• It is a prefix match. The cached part must be identical, byte for byte, from the very first token. Anthropic documents the order as tools, then system, then messages; OpenAI and Gemini also match from the start of the request.
• It is not response caching. The model still generates a fresh answer. Google's caching documentation notes that the model does not distinguish cached tokens from regular input; the cached content simply acts as the start of the prompt.
• Entries expire. Caches live for minutes to hours, and an entry that is not reused in time has to be written again.

Prompt caching compared: OpenAI, Claude and Gemini#

Sources: OpenAI's prompt caching guide, Anthropic's prompt caching documentation, Google's context caching guide and Gemini API pricing, all checked in October 2026. Model lists change often, so read the current page for your model before you rely on a number.

A few details from those pages are easy to miss:

• On OpenAI models before GPT-5.6, cached_tokens is rounded down to a multiple of 128.
• Claude reports input_tokens as only the tokens after the last cache breakpoint. Total input is cache_read_input_tokens + cache_creation_input_tokens + input_tokens.
• Gemini's implicit caching comes with no guarantee. Google says it passes on savings "if your request hits caches".
• Gemini 3.8 Flash input costs $0.75 per million tokens and cached input $0.075 through December 31, 2026; both prices double on January 1, 2027.

What it costs: the break-even math#

When writing to the cache is free, caching can only save money. When it is not, a prefix that is never reused costs more than it would without caching. OpenAI's guide spells out the math for GPT-5.6 and later: writing a prefix once and reusing it once costs 1.35 times its normal input price, against 2 times without caching. Across ten requests, one write and nine reads cost 2.15 times, against 10 times.

The same arithmetic for Claude:

So a 5-minute cache pays off as soon as a prefix is used twice within its lifetime, while a 1-hour cache needs three uses. On Opus 5.5 and Sonnet 5.5 the read multiplier is 0.05, which makes long, frequently reused prefixes even cheaper. Claude Sonnet 5.5 lists $2 per million input tokens, $2.50 for 5-minute writes, $4 for 1-hour writes and $0.10 for cache reads.

Gemini's explicit caches trade a write charge for an hourly storage price. Take a 100,000-token document on Gemini 3.8 Flash: storage at $0.50 per million tokens per hour is $0.05 an hour, and each request that reads the cache pays $0.0075 for those tokens instead of $0.075. One request an hour roughly covers the storage, ignoring the cost of creating the cache. The same document on Gemini 3.1 Pro Preview costs $0.45 an hour to store at $4.50 per million, and each hit saves $0.18, so you need at least three requests an hour.

Structure prompts for cache hits#

Every provider gives the same advice in different words: put what stays the same first and what changes last. For an AI feature in a SaaS product or a browser extension, a cache-friendly request looks like this:

1. Tool definitions, sorted in a fixed order and serialized the same way every time.
2. System instructions, with no dates, user names, request IDs or feature flags in them.
3. Shared reference material: style guides, schemas, few-shot examples, product documentation.
4. Per-user context, such as the text of the page the user is on.
5. The user's question.

The first three parts are shared by every user and are what you want cached. The fourth is cacheable within one conversation: if a user asks three questions about the same page, the page text is a stable prefix for the second and third question. Only the last part changes on every call.

Two practical consequences. First, short system prompts do not benefit: if your shared prefix is 300 tokens, no provider will cache it, and the newest Gemini models need 4,096. Caching pays off when you have a large shared block or long conversations. Second, generate the prompt deterministically. A JSON object serialized with keys in a different order, a tool list built from a Set, or a "current time" line all break the prefix.

Turn prompt caching on in Node.js#

These examples use fetch against the REST APIs, so they work in any Node.js backend without an SDK. Read model names from configuration rather than hard-coding them; our post on LLM model retirements in late 2026 explains why.

Claude: one top-level field#

// claude.mjs: automatic prompt caching with one top-level cache_control field.
export async function askClaude(system, messages) {
  const res = await fetch('https://api.anthropic.com/v1/messages', {
    method: 'POST',
    headers: {
      'x-api-key': process.env.ANTHROPIC_API_KEY,
      'anthropic-version': '2023-06-01',
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      model: process.env.CLAUDE_MODEL,
      max_tokens: 1024,
      // Automatic caching: the breakpoint follows the last cacheable block
      // as the conversation grows. Add "ttl": "1h" for the 1-hour cache.
      cache_control: { type: 'ephemeral' },
      system,
      messages,
    }),
  });
  if (!res.ok) throw new Error(`Claude ${res.status}: ${await res.text()}`);
  return res.json();
}

Automatic caching suits chat-style features. For a large shared block followed by per-request content, place an explicit breakpoint on the last stable block instead, for example system: [{ type: 'text', text: STYLE_GUIDE, cache_control: { type: 'ephemeral' } }]. You can use up to four breakpoints per request. In long conversations, note that Claude looks back at most 20 blocks from a breakpoint for an earlier entry, so add a second breakpoint if the conversation grows past that.

OpenAI: automatic, with a cache key#

// openai.mjs: caching is automatic; a stable key helps routing.
export async function askOpenAI(instructions, userText) {
  const res = await fetch('https://api.openai.com/v1/responses', {
    method: 'POST',
    headers: {
      authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      model: process.env.OPENAI_MODEL,
      input: [
        { role: 'developer', content: instructions },
        { role: 'user', content: userText },
      ],
      // One key per feature and prompt version, not per request.
      prompt_cache_key: 'page-summary-v3',
    }),
  });
  if (!res.ok) throw new Error(`OpenAI ${res.status}: ${await res.text()}`);
  return res.json();
}

On models before GPT-5.6, the key matters for routing: caches live on individual machines, and OpenAI suggests about 15 requests per minute per key, splitting busier traffic across several keys with a stable mapping. On GPT-5.6 and later, the key is optional and keeps separate cache accounting per customer or user, and prompt_cache_options sets the mode and the 30m TTL.

Gemini: implicit by default, explicit for big documents#

Implicit caching needs no code. For a large document that many requests share, create an explicit cache and reference it:

// gemini.mjs: create an explicit cache once, then reuse it by name.
const BASE = 'https://generativelanguage.googleapis.com/v1beta';
const headers = { 'x-goog-api-key': process.env.GEMINI_API_KEY, 'content-type': 'application/json' };

export async function createCache(model, systemText, documentText) {
  const res = await fetch(`${BASE}/cachedContents`, {
    method: 'POST',
    headers,
    body: JSON.stringify({
      model: `models/${model}`,
      systemInstruction: { parts: [{ text: systemText }] },
      contents: [{ role: 'user', parts: [{ text: documentText }] }],
      ttl: '3600s', // storage is billed per hour, so keep it as short as useful
    }),
  });
  if (!res.ok) throw new Error(`Gemini cache ${res.status}: ${await res.text()}`);
  return (await res.json()).name; // e.g. "cachedContents/abc123"
}

export async function askGemini(model, cacheName, question) {
  const res = await fetch(`${BASE}/models/${model}:generateContent`, {
    method: 'POST',
    headers,
    body: JSON.stringify({
      cachedContent: cacheName,
      contents: [{ role: 'user', parts: [{ text: question }] }],
    }),
  });
  if (!res.ok) throw new Error(`Gemini ${res.status}: ${await res.text()}`);
  return res.json();
}

Explicit caching is a beta feature under v1beta. You cannot read back the cached content later, and only the TTL or expiry time can be changed after creation.

Measure your cache hit rate#

The usage block on every response tells you whether caching worked. Normalize it once and log it with the feature name and prompt version:

// cache-usage.mjs: one shape for three providers.
export function cacheUsage(provider, body) {
  if (provider === 'anthropic') {
    const u = body.usage;
    const read = u.cache_read_input_tokens ?? 0;
    const write = u.cache_creation_input_tokens ?? 0;
    return { input: read + write + u.input_tokens, read, write };
  }
  if (provider === 'openai') {
    const d = body.usage.input_tokens_details ?? {};
    return { input: body.usage.input_tokens, read: d.cached_tokens ?? 0, write: d.cache_write_tokens ?? 0 };
  }
  // Gemini: promptTokenCount already includes the cached tokens.
  const m = body.usageMetadata;
  return { input: m.promptTokenCount, read: m.cachedContentTokenCount ?? 0, write: 0 };
}

// Hit rate over a window: share of input tokens served from cache.
export const hitRate = (rows) =>
  rows.reduce((s, r) => s + r.read, 0) / Math.max(1, rows.reduce((s, r) => s + r.input, 0));

Chart the hit rate per feature per day. A healthy chat feature with a large shared prefix often settles at a high, stable share; what matters is the change. If it drops to near zero the day you deploy, someone put something dynamic at the top of the prompt. Send that change to wherever your team already looks, for example with Telegram bot notifications from the same job that computes the rate. OpenAI also offers a Prompt Caching Dashboard and a Prompt Cache Diagnostics tool for its API.

Why cached tokens stay at zero#

This is the most common complaint in developer forums, and the causes repeat:

• The shared prefix is too short. In an OpenAI Developer Community thread, a developer with a 436-token fixed system prompt and a longer dynamic user prompt saw "all three returns with zero cached_tokens". The total input was over 1,024 tokens, but the part that repeated was not. Only the identical prefix counts.
• Something early in the prompt changes. A DEV Community write-up traced a cold Claude cache to one line at the top of the system prompt that printed the current date and time. Move dates, user data and IDs after the cached part.
• Tools or settings changed. On Claude, changing tool definitions invalidates everything; changing tool_choice, adding or removing images, or changing thinking or effort settings invalidates the messages. OpenAI recommends keeping tools stable and restricting them with tool_choice or allowed_tools rather than removing them.
• The entry expired. Claude's 5-minute lifetime counts from the start of the request, and Anthropic notes that time spent streaming a response counts against it. A 4-minute answer leaves about a minute for the follow-up.
• Requests land in different places. OpenAI caches are per machine and not shared across organizations or regions; Claude caches are isolated per organization and workspace. A staging key and a production key never share a cache.
• The model's minimum is higher than you think. Moving to Gemini 3.8 Flash raises the implicit minimum to 4,096 tokens, and older Claude models such as Opus 4.5 and Haiku 4.5 need 4,096 too.

Prompt caching in an extension or SaaS backend#

If your browser extension has an AI feature, the API key belongs on your server, and the extension streams answers from it, as our guide to streaming AI responses in a Chrome extension shows. That server is where caching lives, and it changes a few design choices:

• Keep one prompt template per feature and version it. The version goes into the cache key and your usage logs, so a drop in hit rate maps to a specific change.
• Cache per conversation, not per user. A chat about the current page benefits from caching the page text across turns. A one-off summary of a page nobody else will send rarely does, and with a write charge it can cost more.
• Mind rate limits. OpenAI says cached input tokens still count toward tokens-per-minute limits, while Anthropic says cache hits are not deducted against its rate limits. Our guide to LLM API rate limits covers how to handle 429 errors either way.
• Check data retention. On OpenAI models that support both, organizations with Zero Data Retention default to in_memory caching instead of 24h.

Cost is one half of running an AI feature; the other is whether users are happy with it. When answers get slow or wrong, it shows up in reviews and ratings before it shows up in a ticket. Extenify's extension review monitoring records new reviews, ratings and weekly users for your extension and its competitors on the Chrome Web Store, Edge Add-ons and Firefox Add-ons, and sends a daily alert when something moves. Every account starts with a 7-day free trial, and the pricing page lists the plans.

Frequently asked questions#

Does prompt caching change the model's answers?#

No. Caching changes what you pay and how fast the response starts, not what the model reads. The cached prefix is the same text the model would otherwise process again, and the response is generated fresh each time.

Is prompt caching enabled by default?#

On OpenAI, yes, for supported models. On Gemini 2.5 and newer, implicit caching is on by default. On Claude, you opt in with cache_control, either once at the top level of the request or on specific blocks.

How long does a prompt cache last?#

Claude's default is 5 minutes, refreshed for free each time the entry is used, with a 1-hour option. OpenAI GPT-5.6 and later keep entries for at least 30 minutes after the last use; earlier models use in_memory (about 5 to 10 minutes of inactivity) or 24h retention. Gemini explicit caches last as long as the TTL you set, 1 hour by default.

Why is cached_tokens always 0 in my OpenAI responses?#

Usually because the identical part at the start of your prompt is shorter than the minimum, or because something in it changes between requests. Put your fixed instructions and tools first, keep them byte-identical, and check that they exceed 1,024 tokens on GPT-5.6 and later.

Can other customers read my cached prompt?#

No. OpenAI says caches are not shared across organizations, and Anthropic isolates caches per organization and, on its own API, per workspace. Treat caching as a cost feature, not a storage feature: you cannot read cached content back.

Summary#

Prompt caching rewards one habit: put stable content first and keep it identical. Know your provider's minimum prefix, lifetime and write price, because a cache that is written and never read now costs more on GPT-5.6 and Claude. Use explicit caches on Gemini only when a large document is read several times an hour. Log the cached and written tokens from every response, chart the hit rate per feature, and alert when it falls after a deploy.

Image credits#

• Cover photo: Drawers of nuts and bolts by Topher on the WordPress Photo Directory, CC0.