Batch API economics: when the -50% applies

A FinOps guide by OptimNow · prices from the live OptimToken catalogue

Most large providers now sell a batch tier: submit a file of requests, get the results back asynchronously, pay roughly half the token price. Prompt caching cuts a larger percentage, but only off input tokens. Batch is the discount that applies to output as well, which is why it moves a bill that generation dominates. What it costs instead is latency, and it is not offered on every model. This guide shows where the −50% is real, with numbers from current prices.

What batch pricing is

A batch API is not a faster way to send many requests: it is a slower one. You upload a job (typically a JSONL file of requests), the provider schedules it on spare capacity, and commits to returning results within a window, usually 24 hours. In exchange, both input and output tokens are billed at a discount, typically 50%. The provider wins because batch jobs smooth out their GPU utilization; you win if, and only if, nobody is waiting on the answer.

Which workloads qualify

The test is simple: if nobody needs the result before tomorrow, batch it. That covers a lot of real enterprise volume:

• Invoice and document processing: extraction runs on documents that arrived in a queue, not on a person in a chat window.
• Call and meeting summaries: the recording is finished; the summary can land an hour later.
• Evaluations and backfills: regression suites, re-scoring historical data, enrichment of an existing corpus.
• Content generation at volume: product descriptions, translations, tagging.

What never qualifies: anything interactive. Support chat, coding assistants, agent loops that chain on each other's output: a 24-hour ceiling is meaningless there. Of the 8 use-case presets on the pricing table, 3 are marked batch-eligible for this reason: meeting summary, invoice processing and call summary.

Who publishes batch prices

Here the −50% story gets narrower. Today, 67 of 288 models on OptimToken publish batch rates. A batch tier is not a property of the model, it is a property of the vendor's billing: it clusters in a handful of catalogues and is absent from the rest.

OpenAI, Google, Mistral and Anthropic price at least 30% of their own catalogue at a batch rate. Zhipu (2 of 16), Moonshot (1 of 7) and xAI (1 of 7) publish one or two isolated rows. That is a model decision rather than a billing policy, and should not be read as batch support. Alibaba, DeepSeek and MiniMax publish none at all, across 64 catalogued models between them. For those vendors the strategy is a low list price rather than a discount tier, which is a reasonable answer to the same problem and a very different one to plan around. If your shortlist sits entirely in that last group, batch is not a lever you have.

When a batch rate exists, it is almost always half

The rate itself turns out to be the predictable part. Of the 67 models publishing a full batch tier, 63 price it at exactly 50% off a 30/70 blend of input and output. The median is 50%. Unlike prompt-cache rates, which spread across the whole range, batch pricing has converged on one number.

The exceptions are worth knowing before you write a migration off the headline. The check is one subtraction against the list price, and it belongs in the migration plan rather than after it.

Worked example: invoice processing at 100K requests/month

Using the Invoice Processing preset (1,500 input + 600 output tokens per request, an async workload), at 100,000 requests per month:

Two lessons. First, when batch rates exist, the saving is real money at volume: no code changes to the prompts, just a different submission path. Second, a cheap model at list price can still undercut an expensive model at batch price. DeepSeek V4 Flash at list ($0.02/M in, $0.32/M out) costs a fraction of Claude Sonnet 5's batch rate for this extraction-style workload. Batch discounts are an optimization within a model choice, not a substitute for making the model choice well. Compare the two side by side before assuming the discount settles it.

The caveats that bite in production

• The SLA caps the wait; it makes no promise about how fast a job finishes. Jobs often complete in minutes, but the only guarantee is the window (typically 24h). Downstream steps must tolerate the worst case.
• No streaming. Results arrive as a completed file. If your pipeline expects token streams, it needs a second, non-streaming path.
• Job size and queue limits. Providers cap requests per batch and enqueued tokens per model. A month-end invoice spike can hit those caps exactly when you need throughput.
• Results expire. Output files are retained for a limited period: collection must be automated, not manual.
• Batch × cache interaction is murky. Prompt-cache discounts are published for the standard tier; few providers publish a batch-tier cache-read rate. The after-discounts column on the pricing table assumes the standard cache rate under batch, which is the conservative reading (the formula is here).

When not to bother

Skip batch when the workload is interactive, or when a deadline sits closer than the SLA window. Skip it too when the monthly spend is small. An engineer-day of pipeline work does not pay back on a $40/month job, and at that scale a budget model at list price is usually the better lever. And never batch-migrate a workload before checking whether a cheaper model would beat the discount outright. The pricing table sorts by optimized cost per use case for exactly that question.

Figures on this page come from the 2026-09-28 price snapshot and rebuild nightly.

The FinOps reasoning behind this guide comes from OptimNow's open-source practice library: github.com/OptimNow/cloud-finops-skills. If you want your own async workloads sized against the batch and cache rates you are entitled to, OptimNow runs that review.

Data snapshot: 2026-09-28 · Compare all models · Cloud compute pricing · All guides