Module 3 of 5 · Intermediate · 8 min read
Cost, speed and quality
Token pricing, latency and throughput, and how to estimate what a workload will cost.
- 1Prices are per million input and output tokens; output usually costs 3-5x more.
- 2Reasoning tokens count as output and can dominate cost.
- 3Caching, batching and smaller models cut costs by 50-90%.
01How pricing works
API prices are quoted per million tokens, separately for input (what you send) and output (what the model writes). A million tokens is roughly 750,000 words. Prices range from a few cents for small models to tens of dollars for flagships.
Reasoning models also produce hidden thinking tokens billed as output. A hard question can use thousands of them.
02Speed has two parts
Latency (time to first token) decides how responsive a chat feels. Throughput (tokens per second) decides how long a long answer takes. Reasoning adds thinking time before the first visible token.
For interactive products, a fast model that answers in under a second often beats a smarter one that thinks for ten.
03Cutting costs
Most teams overspend by using the flagship for everything.
- Prompt caching: reuse long, repeated context at a steep discount.
- Batch APIs: around half price for work that can wait hours.
- Routing: send easy requests to a small model and hard ones to a large one.
- Effort settings: lower reasoning effort for simple tasks.
Finished reading?
Mark it complete to track your progress, then test yourself with the Models & Labs quiz.