Prompt caching: stop paying full price for the same prefix

Most production prompts are mostly identical from one call to the next: the same long system prompt, the same tool definitions, the same retrieved context — with only a short new user turn at the end. Prompt caching lets the provider reuse the processing of that stable prefix, so you're not billed full price to re-read the same thousands of tokens on every request.

Why it saves so much

The cached prefix is billed at a steep discount (and served faster) on every call after the first. For apps with big system prompts or long shared context — which is most serious ones — that's a large, quiet cut to both cost and latency, with zero effect on output quality. Put the stable stuff first and the changing stuff last, and caching does the rest.

How the big three differ

• OpenAI — automatic. Prompts over a length threshold are cached transparently, and you get the discount on repeated prefixes without changing your code.
• Anthropic — explicit. You mark where the cacheable prefix ends with a cache_control breakpoint, which gives you precise control over what's cached.
• Google Gemini — explicit context caching. You create a cached-content handle for a large shared context and reference it across requests, ideal for asking many questions of the same big document.

const res = await anthropic.messages.create({
  model: 'claude-sonnet-4-20250514',
  system: [
    { type: 'text', text: LONG_STABLE_INSTRUCTIONS,
      cache_control: { type: 'ephemeral' } },   // cache up to here
  ],
  messages: [{ role: 'user', content: newUserTurn }],
});

Design for it

Caching rewards a stable, front-loaded prompt. Keep the unchanging prefix — instructions, examples, tools, shared context — at the top and byte-for-byte identical between calls, and put anything that varies at the end. Structure the prompt for the cache and the savings are close to free.

You wrote the system prompt once. Prompt caching means you stop paying to process it a thousand times.