Shrinking the Token Budget Without Shrinking the Team

The dominant response to rising AI costs has been to cut people and call it efficiency. It treats payroll as the flexible line in the budget and the token bill as fixed. That is backwards. The token bill is the most engineerable cost in the entire AI stack, and it bends by 50 to 90 percent using techniques that are already standard. Payroll cuts move in one direction, take institutional knowledge with them, and on the evidence so far do not improve returns. The disciplined move is to engineer the token budget down and reinvest the difference in the people who make the tokens worth anything.

Key takeaways

Most AI-era layoffs are financing, not efficiency, and independent research finds they do not correlate with better returns.
The token bill is the most flexible cost in the AI stack, reducible 50 to 90 percent through caching, routing, batching, and retrieval.
Prompt caching cuts repeated-input cost by up to 90 percent, and one team cut total spend 59 to 70 percent by raising its cache hit rate.
Payroll cuts are largely irreversible and remove the junior pipeline that produces the senior engineers these systems require.
Treat token spend as an engineering metric with per-team budgets, and reinvest recovered budget in amplifying the workforce.

Why the token bill, not the payroll, is the flexible line

Leaders have been cutting headcount to fund AI while treating the token bill as a fixed cost of doing business. The order should be reversed. Nvidia's Jensen Huang framed heavy token spend as a virtue, going so far as to say a $500,000 engineer whose annual token use came in under half their salary would concern him, and the largest hyperscalers have guided roughly $700 billion in 2026 capital expenditure. The spending has not delivered what it promised. Gartner surveyed 350 executives at companies above $1 billion in revenue and found roughly 80 percent had cut headcount with no correlation to improved returns. Its analyst Helen Poitevin concluded that workforce reductions may create budget room but do not create return. Payroll is the wrong line to treat as elastic: cuts happen once, they are slow and costly to reverse, and they remove knowledge the remaining systems still depend on. The token bill has none of those properties.

Where the token budget actually bends

The token bill is engineerable, and most organizations have not engineered it. The largest and least glamorous lever is to stop paying to process the same text repeatedly. Prompt caching, now standard across the major providers, charges cached input at a fraction of the base rate: Anthropic reads from cache at one tenth of the input price, a 90 percent discount, and OpenAI applies a comparable discount on cached input automatically above 1,024 tokens. The effect in production is not theoretical. The security firm ProjectDiscovery raised its cache hit rate from 7 to 84 percent by moving dynamic content out of the cached prefix, cutting total LLM spend 59 to 70 percent while serving 9.8 billion tokens from cache. That single engineering exercise recovered more budget than most AI-attributed layoff rounds save.

The other levers compound. Route work to the right-sized model instead of sending routine classification and summarisation to the most expensive tier by default. Move anything that does not need a real-time answer to batch processing, which carries a 50 percent discount. Use retrieval to send the model only the relevant slice of a knowledge base rather than the whole thing, compress prompts to strip redundant context, and run open-weight models for routine workloads where the infrastructure is worth managing. None of this requires a single layoff, and all of it is reversible if a workload's economics change. The trade-offs are ordinary engineering decisions: caching and routing add operational complexity, batch trades latency for price, and open-weight shifts cost from API bills to infrastructure. Each has measurable payback, which is the point.

The savings only matter if they fund the right work

Cost optimization is not the goal. It frees budget, and where that budget goes decides the return. The same Gartner research found the organizations that improved returns were those using AI to amplify their workforce rather than replace it. Klarna ran the experiment in public: it replaced roughly 700 customer service roles with an AI assistant, watched quality fall, and reversed into a blended model where AI absorbs routine volume and people handle judgment. Its chief executive summarised the AI-only result as lower quality and not sustainable. Uber hit the same boundary from the cost side, exhausting its 2026 AI budget by April after equipping 5,000 engineers with coding tools, with the connection to anything customers notice still absent. AI pays off when it is engineered into a measurable operating system, the way a call-to-deal attribution pipeline or a product-aware recovery system produces value you can point to, not when it is treated as a headcount substitute.

The reinvestment that is urgent, not optional

There is one place the recovered budget should go first, because the alternative compounds. Stanford's Digital Economy Lab, analysing payroll records for millions of workers, found employment for software developers aged 22 to 25 fell nearly 20 percent from late 2022 while older cohorts grew. Cutting entry-level roles removes the training ground for the senior engineers who will direct these systems in five years. A business that has just engineered 50 to 70 percent off its token bill has the room to keep hiring at the bottom rung. Whether it does is a leadership decision, not a financial constraint.

Make token spend an engineering metric

The discipline that separates the two outcomes is measurement. Track cache hit rate as a first-class production metric, set per-team token budgets before a runaway bill forces the issue, and measure the operating value a workload produces before deciding what to cut. Uber imposed a $1,500 monthly cap per engineer only after its overrun. The companies that come out ahead choose that discipline before the budget forces it on them.