← OptimToken

Batch API economics: when the -50% actually applies

A FinOps guide by OptimNow · prices from the live OptimToken catalogue

Most large providers now sell a batch tier: submit a file of requests, get the results back asynchronously, pay roughly half the token price. On paper that is the single biggest published discount in LLM pricing — bigger than most negotiated enterprise discounts. In practice it applies to a narrower slice of workloads than the marketing suggests, and for some models it is not available at all. This guide shows where the −50% is real, with numbers from the current catalogue.

What batch pricing actually is

A batch API is not a faster way to send many requests — it is a slower one. You upload a job (typically a JSONL file of requests), the provider schedules it on spare capacity, and commits to returning results within a window, usually 24 hours. In exchange, both input and output tokens are billed at a discount, typically 50%. The provider wins because batch jobs smooth out their GPU utilization; you win if, and only if, nobody is waiting on the answer.

Which workloads qualify

The test is simple: would anyone notice if the result arrived tonight instead of now? If no — batch it. That covers a lot of real enterprise volume:

• Invoice and document processing — extraction runs on documents that arrived in a queue, not on a person in a chat window.
• Call and meeting summaries — the recording is finished; the summary can land an hour later.
• Evaluations and backfills — regression suites, re-scoring historical data, enrichment of an existing corpus.
• Content generation at volume — product descriptions, translations, tagging.

What never qualifies: anything interactive. Support chat, coding assistants, agent loops that chain on each other's output — a 24-hour ceiling is meaningless there. In the hub's eight business use-case presets, three are modeled as batch-eligible for exactly this reason: invoice processing, call summaries, and meeting summaries.

Who actually publishes batch prices

Here the −50% story gets narrower. In the current OptimToken snapshot, 58 of 259 catalogued models publish batch rates. OpenAI and Anthropic list them for their main model families. Many budget and open-weights providers publish none — their strategy is a low list price rather than a discount tier. That asymmetry matters for model selection, as the third row below shows.

Worked example: invoice processing at 100K requests/month

Using the hub's invoice-processing profile (1,500 input + 600 output tokens per request, an async workload), at 100,000 requests per month:

Two lessons. First, when batch rates exist, the saving is real money at volume — no code changes to the prompts, just a different submission path. Second, a cheap model at list price can still undercut an expensive model at batch price — DeepSeek V4 Flash at list ($0.14/M in, $0.28/M out) costs a fraction of Claude Sonnet 5's batch rate for this extraction-style workload. Batch discounts are an optimization within a model choice, not a substitute for making the model choice well. Compare the two side by side before assuming the discount settles it.

The caveats that bite in production

• The SLA is a ceiling, not a promise of speed. Jobs often complete in minutes, but the only guarantee is the window (typically 24h). Downstream steps must tolerate the worst case.
• No streaming. Results arrive as a completed file. If your pipeline expects token streams, it needs a second, non-streaming path.
• Job size and queue limits. Providers cap requests per batch and enqueued tokens per model. A month-end invoice spike can hit those caps exactly when you need throughput.
• Results expire. Output files are retained for a limited period — collection must be automated, not manual.
• Batch × cache interaction is murky. Prompt-cache discounts are published for the standard tier; few providers publish a batch-tier cache-read rate. The hub's optimized cost deliberately applies the standard cache rate under batch — the conservative assumption.

When not to bother

Skip batch when the workload is interactive, when monthly spend on the workload is small (an engineer-day of pipeline work does not pay back on a $40/month job — and at that scale a budget model at list price is usually the better lever anyway), or when a deadline sits closer than the SLA window. And never batch-migrate a workload before checking whether a cheaper model would beat the discount outright — the pricing table sorts by optimized cost per use case for exactly that question.

Data snapshot: 2026-08-14 · Compare all models · Cloud compute pricing