Prompt caching: the discount nobody budgets for

Published 2026-08-16 · Prose last reviewed 2026-10-20 · Figures recomputed nightly from the 2026-11-14 catalogue snapshot

A FinOps guide by OptimNow · prices from the live OptimToken catalogue

Of the discounts published on an LLM rate card, prompt caching carries the largest percentage. Across the 226 models on OptimToken that publish a cache-read price, the median is 88% off input. It is also the narrowest: it applies to input tokens only, and only to the share of them that repeats. At the hit rates modelled for the 8 use-case presets on the pricing table, the median saving that reaches an invoice runs from 3.3% on Call Summary to 21.0% on Knowledge Q&A.

The discount is not one number

Rate cards do not agree on what a cache read costs. The 3 most common published rates are 10%, 20% and 25% of the list input price, and the full range runs from 0% off to 99% off. Of the 312 models, 86 publish no cache-read price at all, which in practice means no cache discount to plan around.

Two consequences for model selection. A cache-heavy architecture is worth more on some models than others, so the cache rate belongs in the comparison alongside input and output. And a switch between two similarly priced models can change the cache economics without changing the list price at all.

Why the headline never reaches the bill

A cache read discounts input tokens. It does nothing to output tokens, and output is where most of the money goes on a typical workload. Two multipliers sit between the headline and the invoice: the share of the bill that is input, and the share of input that is a cache hit rather than a fresh read.

The ceiling matters as much as the realised figure. Even at a perfect hit rate, the median saving tops out at 56.3% on Meeting Summary, the most input-heavy profile, and sits at 19.6% on Marketing Content, where long outputs dominate. The 88% is a discount on input tokens, not on the bill, and those are different budgets.

Worked example: an agent workflow at 100,000 requests/month

The Agent Workflow preset models 6,000 input and 3,000 output tokens per request, at a 70% cache hit rate. Agent loops re-send a large stable prefix (system prompt, tool definitions, prior turns) on every call, which is the pattern caching was designed for.

On Claude Opus 5, caching returns $1.9K a month at this volume, against a list rate of $5.00/M input and $25.00/M output. A saving of that size pays for a caching project at this volume. It is also 18.0% of the bill, not 90%. GPT-5.5 returns the same $1.9K, from the same input price and the same 90% cache rate, but against a larger bill, so it lands at 15.7%. The cache rate is identical on both rows. The output price decides what it is worth. Compare the two side by side before assuming the cache rate settles the choice.

The same model, a workload where it does not pay

Hold the model still and change the workload. On the Marketing Content profile (2,500 in, 1,800 out, 20% hit rate), the same Claude Opus 5 saves 3.9%: $225 a month instead of $1.9K. The cache rate did not change. The workload writes long outputs from short, mostly fresh prompts, so there is little input to discount and little of it repeats. Generation-heavy work with varied prompts is the case where a caching project does not repay the effort of building it.

The cache price that saves nothing

For 6 models the catalogue quotes a cache-read price identical to the list input price, among them gpt-oss-20b, Schematron V2 Turbo, Schematron V2 Small, Qwen3 30B A3B Instruct 2507. The quoted price is one host's rate, and on these rows that host lists the two fields at one figure. The field is populated, so a spreadsheet that scores a model on whether a cache price exists scores these as cache-friendly. The discount is 0%. Compare the cache rate against the input rate, and never test for the presence of a rate.

The write premium, which most business cases leave out

Every figure above uses published read prices only, so each is a ceiling on the saving rather than the saving. Writing to the cache is billed separately, and above the standard input rate. Business cases leave the write out, and it decides whether caching pays at all.

Anthropic publishes ×1.25 on input for a 5-minute entry and ×2 on input for a 1-hour entry. Other model makers structure it differently, and some fold the write into the standard input price, so treat these as Anthropic's published terms rather than a market rate. They are the only 2 numbers on this page not computed from the catalogue.

The hub's own Cost / 1K req (after discounts) column does charge the write, which is why it can sit slightly above the read-only figures on this page. It assumes one token written for every 5 read. The write is billed at the model's published write rate where there is one, and at the ordinary input rate where there is not (the full formula).

The premium sets a break-even: a cached prefix has to be read enough times inside its lifetime to earn back what the write cost. Against the read rates published today:

Read the table as a floor, not a target. At the least favourable rate in the catalogue (2 models at 95% of input), 22 calls sharing one prefix are enough; at the 3 most common rates, 3 calls. On a high-traffic path the break-even is reached in the first minute.

It also names the failure case exactly. A workload whose calls arrive further apart than the cache lifetime never reaches the second read. Every call writes a fresh entry, and the prefix bills at ×1.25 or ×2 of input instead of ×1. Cached, that endpoint costs more than leaving it alone. The variables are request rate and prefix stability, not prompt size: a large prompt cached once an hour loses money, and a small one cached every second makes it.

What to check before budgeting a caching saving

• Compute the input share of your own bill first. Caching cannot save more than that share, whatever the discount. On the Marketing Content profile the median model spends 74% of the request on output, so the ceiling is set before caching enters the discussion.
• Measure the hit rate, do not assume it. The gap between the two bars in the chart above is entirely hit rate. The API reports cache-read and cache-write tokens separately; that ratio is the number to track.
• Check the prefix is stable byte for byte. A timestamp, a session ID or a reordered tool list at the top of a prompt invalidates the entry. This is the most common reason a measured hit rate lands far below the modelled one.
• Match the cache tier to the traffic pattern. The longer tier costs twice the input rate to write. It pays only when calls are spread out enough to need it and frequent enough to reach break-even.
• Recheck after any model switch. Published discounts run from 0% to 99% across this catalogue, so a migration can change the caching business case even when list prices look similar.

For an organisation early in its FinOps practice, the useful first step is measurement rather than optimisation: split the bill into input and output, then into cache reads and writes. Teams already tracking that split can move to the break-even test per endpoint. These figures are list prices from the current snapshot, not audited spend, and they assume the token presets on the pricing table rather than your own traffic. Batch is the other published discount, and the one that reaches output tokens: when the −50% applies covers it.

The FinOps reasoning behind this guide comes from OptimNow's open-source practice library: github.com/OptimNow/cloud-finops-skills.

Data snapshot: 2026-11-14 · Compare all models · Cloud compute pricing · All guides