There’s a meeting that kills more agent projects than any bug ever has. It happens about two months after launch, and it isn’t run by engineering. Finance opens a dashboard, points at a line that grew 8x while usage grew 2x, and asks a question nobody in the room can answer: “What are we paying for, exactly?”
The engineers aren’t incompetent — the system works. But nobody designed its cost behavior, so the cost behavior designed itself. This post names the discipline that prevents that meeting: token economics. It’s the sixth pillar in the arc I’ve been writing — after context engineering, spec-driven development, evals, OpenSpec and durable execution — and it’s the one that decides whether the other five get to keep running in production.
What is token economics?
Token economics is the engineering discipline of treating LLM cost as a designed property of the system rather than a discovered one. It covers four decisions: what each request is allowed to cost (budgeting), what work you refuse to pay for twice (caching), what work can wait for cheaper processing (batching), and which model earns each request (routing). Teams that make these decisions explicitly ship agents with predictable unit costs; teams that don’t find out their unit costs from the invoice.
The word “economics” is doing real work in that definition. This isn’t cost cutting — it’s cost modeling: knowing the marginal cost of one more user, one more feature, one more agent step, before finance asks.
Nobody designed its cost behaviour, so the cost behaviour designed itself. That sentence describes almost every agent system that gets shut down in its second quarter.
There are six levers, and they are worth knowing as a set, because each one has a failure mode that shows up as a cost surprise rather than an error:
| Lever | What it does | Typical effect | How it backfires |
|---|---|---|---|
| Per-request budget | Caps what one run may spend, enforced at runtime | Turns “can we add a verification step?” from a debate into arithmetic | Sized from averages, so the long-tail runs die at the ceiling instead of finishing |
| Prompt caching | Stops re-buying the stable prefix on every step | Large cut in paid input tokens, with no quality tradeoff at all | A prompt assembled in a varying order misses silently — full price, no error |
| Batch processing | Moves work nobody is waiting for to discounted async | Steep discount on evals, backfills and summarisation | Applied to something a user is waiting for, and now the feature feels broken |
| Model routing | Sends each step to the cheapest model whose output you can verify | Most steps get cheap; tokens concentrate where quality binds | Done without evals, so you optimise blind and ship a quality regression |
| Checkpointing | Lets a failed run resume instead of restart | Retries stop re-buying the whole conversation history | Checkpoints too coarse, so you replay most of the run anyway |
| Context eviction | Drops what the next step demonstrably doesn’t need | Stops cost growing quadratically with run length | Evicts the one constraint the agent needed to keep obeying |
Why do agent costs explode when request costs don’t?
Because agents multiply. A chat feature costs roughly one model call per user action — linear, boring, safe. An agent takes steps, and each step can carry the accumulated context of every step before it. Cost per run isn’t step-cost × steps; it’s closer to step-cost × steps², because the context window grows as the run gets longer. Add retries, tool-result payloads dumped verbatim into context, and a planner that “thinks about it one more time,” and an 8x cost surprise on 2x usage stops being mysterious.
This is why token economics is inseparable from context engineering. Every context-engineering decision — what to retrieve, what to summarize, what to evict — is also a purchasing decision. A sloppy context strategy isn’t just a quality problem; it’s a standing order to buy the same tokens over and over.
The durable-execution connection is just as direct: an agent that can resume instead of restart doesn’t re-buy its own history after every failure. In cost terms, checkpointing is a discount program. Retry-from-zero architectures pay full price for the same work N times and call it reliability.
What should a token budget actually look like?
A real token budget is a per-request ceiling with an owner and an enforcement point, not a monthly total on a dashboard. The working form: each feature declares its expected cost per request and its hard maximum, the system enforces the maximum at runtime (an agent that hits its ceiling stops and reports, exactly like a timeout), and someone named owns the number. Monthly totals discover problems; per-request ceilings prevent them.
Budgets change engineering conversations in a way dashboards never do. “Can we add a verification step to the agent?” stops being a philosophical debate about quality and becomes arithmetic: the step costs ~4k tokens per run, the budget has 3k of headroom, so either the step gets cheaper or something else does. That’s a productive argument. Teams have it before shipping instead of after the invoice.
The budget also has to be visible in the same place the spec lives. If you practice spec-driven development, the cost ceiling belongs in the spec next to the behavioral requirements — “answers within 30 seconds, costs under X per run” — because a requirement nobody wrote down is a requirement the system doesn’t have.
What does caching buy you, really?
Prompt caching is the highest-leverage cost tool in the stack because agent workloads are extraordinarily repetitive: the system prompt, the tool schemas, and the long-lived context are identical across steps and often across users. Cache-aware request design — stable prefix first, volatile content last — routinely cuts the paid token volume of an agent system by half or more, with no quality tradeoff whatsoever. It’s the closest thing to free money in AI engineering.
But caching only pays if requests are built to be cacheable, and that’s an architecture decision, not a flag. A prompt assembled in a different order per request defeats the cache silently — you pay full price and nothing errors. This is a place where the difference between “read the API docs once” and “actually knows the platform” shows up directly on the invoice; the mechanics (cache breakpoints, prefix design, TTLs, and how to verify you’re getting cache hits) are exactly the kind of thing the Claude API course drills, because verified savings beat assumed savings.
Designing a request that can actually be cached
The rule is simple to state and easy to violate: everything stable goes first, everything volatile goes last. System instructions, tool schemas, and long-lived project context form the prefix; the user’s current message, retrieved chunks for this query, and step-specific state come after. The moment one varying token appears early, everything after it becomes uncacheable — the cache matches on prefixes, not on content it recognises later.
The self-inflicted wound I have seen most often is a timestamp. Someone adds a helpful Current time: … line near the top of the system prompt, and the entire prefix changes on every single call. The prompt still works. The evals still pass. The bill quietly doubles, and there is no failure anywhere to investigate. The same goes for a user’s name, a request ID, or a randomly-ordered set of tool definitions assembled from a hash map — anything that varies per call belongs after the stable block, or nowhere.
Cache lifetime matters too. Cached prefixes expire, so a low-traffic feature may pay to write the cache and never read it before it lapses; a high-traffic one amortises the write across thousands of calls. This is why caching wins so decisively on agent loops specifically — a single run makes many calls in quick succession against an almost identical prefix.
Verifying the savings instead of assuming them
Log cached and uncached input tokens as separate fields, always. A dashboard showing one merged “input tokens” number cannot tell you whether caching is working, and teams routinely believe they are getting a discount they stopped getting weeks ago. With the split recorded, the derived metric you actually want is a cache-hit ratio per feature, and the alert you want is a drop in that ratio — because the thing that breaks caching is never a deploy labelled “break the cache,” it’s an innocuous refactor that reordered how the prompt gets assembled. That alert is the difference between finding out in an afternoon and finding out in a quarterly review.
Batching is caching’s quieter sibling: any work that doesn’t need an answer in seconds — nightly summarization, backfills, eval suites — can run through batch processing at a steep discount. The design question is simply which of your workloads are secretly asynchronous. In my experience it’s more of them than the architecture admits, evals being the canonical example: a comprehensive suite run nightly at batch prices costs less than a thin one run synchronously in panic after an incident.
When should a smaller model take the request?
Route by verifiability, not by vibes: use the smallest model whose output you can check, and reserve the frontier model for steps where being wrong is expensive and checking is hard. Classification, extraction, formatting, summarization-for-storage — these have verifiable outputs and tolerate a cheaper model backed by validation. Planning, tool selection in open-ended situations, and final user-facing synthesis usually earn the big model. A well-routed agent often runs most of its steps on small models while spending most of its tokens where quality actually binds.
Routing is also where evals stop being optional. Without a regression suite, every routing change is a leap of faith, and teams either never optimize (expensive) or optimize blind (worse). With a real eval harness, “move extraction to the small model” is an afternoon’s experiment with a pass/fail answer. The eval suite is what makes cost optimization safe — that’s the trinity again: spec says what must hold, evals verify it still does, and token economics decides what it may cost.
The pattern that beats both extremes is escalation rather than assignment: run the step on the small model, validate the output mechanically, and retry on the frontier model only when validation fails. If the cheap model is right eighty percent of the time and validation is reliable, you pay small-model prices four times out of five and still get frontier-model correctness — better economics than routing everything down, and better quality than routing everything up.
A concrete shape, from a support-triage agent I have worked on: classify the incoming request (small model, output is one of a fixed set of labels, trivially checkable), retrieve the relevant history (no model at all — this is a database query that teams routinely hand to an LLM for no reason), draft the response (mid-tier), then final-check for policy and tone before it reaches a human (frontier). Four steps, three different models, one step with no model. Most of the steps run cheap; most of the tokens land on the two that matter.
The anti-pattern worth naming: routing by customer tier instead of by task. Paying customers do not need a bigger model, they need a correct answer — and a cheap model with validation often produces one more reliably than an expensive model with none. Route by verifiability, let the evals decide, and keep the pricing page out of the architecture.
For multi-agent systems, add one rule: sub-agents don’t inherit the orchestrator’s model by default. Each role gets the cheapest model that passes that role’s evals — the difference between a fleet that scales and a fleet that gets shut down is usually this rule, applied early. Building agents with per-role model choices and cost instrumentation from day one is a core pattern in the Agent SDK course, for exactly this reason.
How do you make cost visible before finance does?
Instrument cost per request, per feature, and per agent-step from the first deploy — the same way you treat latency. Log tokens in and out (cached and uncached separately, or you’ll fool yourself), tag by feature and step, alert on per-request outliers rather than monthly totals. One slow query doesn’t wait for the monthly infrastructure review; one 50x-cost agent run shouldn’t either.
Five fields per model call are enough to answer almost every question you will later be asked: feature, step, model, input tokens split into cached and uncached, and output tokens. Everything useful derives from those — cost per run, cost per feature, cache-hit ratio, the distribution that tells you whether your budget ceiling is sized for the average or the tail. Teams that skip the step tag are the ones who know their agent is expensive but cannot say which of its eight steps is responsible, which turns a one-afternoon fix into a week of guessing.
Then two alerts, and resist adding more. First, a per-request cost outlier — a single run crossing some multiple of its budget, because that is what a runaway loop looks like from the outside. Second, a drop in cache-hit ratio per feature, which is how the silent doubling announces itself. Monthly-total alerts feel responsible and are nearly useless: by the time a monthly number is alarming, the money is spent and the cause is four deploys back.
The reporting artifact that changes the politics is the unit-cost curve: cost per successful task over time. It reframes the finance meeting from “why is the bill growing” (a threat) to “unit cost fell 40% while usage doubled” (a victory lap). Same numbers, opposite meeting — and the team that owns its unit costs gets to keep experimenting, because leadership trusts the meter is being watched.
The discipline, compressed
Give every request a budget with an owner. Build prompts to be cached, and verify the hits. Batch everything that’s secretly asynchronous. Route by verifiability, with evals as the safety net. Checkpoint so retries don’t re-buy history. Instrument unit cost from day one, and report it before you’re asked. None of this is exotic — it’s the same maturity curve infrastructure cost went through a decade ago, compressed into a year. The teams that internalize it get to run ambitious agent systems; the teams that don’t get the meeting. The RAG-and-agents architecture patterns all assume this discipline underneath — and if you want the operating-model version of the whole arc, spec-driven development foundations is where the six disciplines get assembled into one workflow.
