Your agent's biggest bill isn't the model — it's the context you re-send

Here's the arithmetic nobody runs before their first big agent bill. An agent loop is a conversation the model never leaves: every turn, the API receives the entire history again — system prompt, tool definitions, every file it read, every diff it produced. A 24-turn run with a context that grows to 80K tokens doesn't process 80K tokens. It processes something like a million, because turn after turn, the same tokens ride the wire again and get re-read at full price.

The model tier you picked is a linear cost factor. The loop is a quadratic one. That's why two operators running the same model on the same task can see bills an order of magnitude apart — and why the cheapest lever in agent operations has nothing to do with switching models.

The lever: a stable, cacheable prefix

Every serious provider now offers prompt caching: if the opening portion of your request is byte-identical to a recent one, those tokens are served from cache at a fraction of the price (on Anthropic's API, cached reads cost 10% of fresh input). An agent loop is the perfect customer for this — if the harness is built for it. The requirement is simple and brutal:

The stable-prefix checklist:

• System prompt: frozen. No clock, no run ID, no per-task text at the top. Task-specific content belongs in the first user message, after the cacheable region.
• Tool definitions: stable and ordered. Same tools, same order, same schemas, every turn. Generating tool lists dynamically is a silent cache-killer.
• Append, don't rewrite. The conversation should only grow at the end. Anything that edits earlier turns (re-summarizing history in place, rotating context) invalidates everything after the edit.

The second habit: stop hoarding context

Caching makes re-sending cheap; it doesn't make it free, and it doesn't help the model think. The other half of context economics is refusing to carry what the run no longer needs. The pattern that works is just-in-time retrieval: give the agent tools to find information (search, read, list) instead of pre-loading everything it might conceivably want. A lean context with good retrieval beats a stuffed context every time — on cost, and usually on accuracy, because models attend better when there's less noise competing for attention.

Long runs also need compaction: when the history approaches the window budget, summarize the middle, keep the head (your cacheable prefix) and the recent tail intact. Good harnesses do this automatically. If yours doesn't, the run doesn't just get expensive — it eventually gets stupid, as the window fills with stale tool output the model dutifully re-reads instead of thinking.

The third habit: route by task, cap by budget

Not every step in a run deserves the frontier model. Planning, reviewing, and debugging reward the expensive tier; mechanical steps — renames, formatting fixes, applying a described change — don't. Harnesses that route steps to model tiers save real money without measurable quality loss. And every run should carry a hard budget stop — in tokens and dollars — because "the loop got stuck retrying" is the most common way an unattended agent turns into a surprise invoice. A stop condition isn't pessimism. It's the same discipline as a stop-loss order: you set it when you're calm, so it triggers when you're not looking.

What this looks like in practice

1. Freeze your system prompt and tool list; move anything dynamic below them.
2. Confirm cache hits are actually happening (your provider reports cached vs. fresh input tokens — read that field once and you'll never ignore it again).
3. Prefer retrieval tools over pre-loaded context; let compaction handle long runs.
4. Set a per-run budget in dollars. Have the harness enforce it, not your attention.

None of this requires new infrastructure. It's operator discipline — the difference between renting a fast intern and running one professionally.