Original Title: Token Budget Wars
Original Author: Jaya Gupta
Original Translator/Compiler: Peggy
Editor's Note: Enterprise AI is moving from the stage of 'whether to adopt' to the stage of 'how to account for it'.
Over the past two years, many companies pushed employees to use AI, more to keep up with technological trends and competitive pressures. But when AI inference costs shift from experimental budgets to ongoing operational expenses, CEOs and CFOs are beginning to ask a more practical question: How much value is AI actually creating? What tangible outcome is gained for each dollar of token cost?
This is the heart of the 'Token Budget Wars.' The so-called token budget war is not just about companies wanting to lower their AI bills; it's about reassessing which business areas deserve more computing power, which tasks should be switched to cheaper models, which processes can be outsourced or done manually, and which are just wasteful consumption.
The most notable point of the article is that AI usage volume does not equate to value. In the SaaS era, usage typically indicated software adoption; but in the AI era, token consumption only tells us 'the meter is running.' The same workflow can incur costs that differ by multiples due to variations in prompts, context, model selection, and retry counts. A higher bill could mean AI is genuinely doing work, or it could mean the system is wasting effort.
Therefore, the next phase of enterprise AI hinges not just on model capability, but on the ability to correlate token costs with business outcomes. The first phase proved AI can perform tasks; the second phase must answer: Are these tasks worth paying for?
Below is the original text:
Enterprise AI Has Shifted from 'Whether to Adopt' to 'How to Allocate.'
In the corporate C-suite, the new 'currency' is your ability to quantify the ROI of AI investment. Every functional department is being asked the same question: What did you produce? What was the cost? For the past two years, CEOs, while waking up to Jim Cramer on CNBC (#bearish) and watching competitors announce productivity gains, have demanded their companies use AI across the board. The real pressure now comes from the follow-up question: Show me the proof of value.
Claude was launched in November 2025, by which time most enterprises' 2026 annual budgets were already locked. By the first quarter, actual enterprise usage far exceeded original plans. Inference costs are no longer just a line item for experimentation but have become an ongoing operational expense. This brings a new question: Where is AI actually creating value?
This question is difficult to answer because the utility of tokens is not quantified. The bill doesn't tell you whether this expense replaced labor, generated revenue, reduced risk, accelerated a process, or was just a group of engineers spamming tokens for a leaderboard (#metamates). When spending is only a few hundred thousand dollars, it still looks like an experiment. But beyond a certain threshold, say seven figures, it becomes infrastructure. Technical differences begin to materially impact the P&L: the same workflow, the same inputs, could have token costs 5 to 10 times different between two runs, with no apparent surface-level issue. At experimental scale, this variance is expensive; at infrastructure scale, it becomes a number the CFO must explain to the CEO.
Call it 'marginal token utility': the business value created per additional dollar of inference cost. This is the number that truly matters at scale, and the one most companies currently cannot see.
Boardroom questions are shifting from 'Is AI useful?' to 'Where is AI actually providing leverage?' Precisely because of this, the so-called token budget war is essentially a battle over the allocation of tokens.
The fight over token ownership is heating up quickly because it collides with a thirty-year-old executive instinct: a large team means a big title, large scope of responsibility, and greater power. In the past, the visible marker of a senior executive's success was the size of their team—direct reports, indirect reports, headcount on the org chart.
But when intelligence becomes the scarce resource, the new marker becomes: how much intelligence you can command.
AI spending is essentially competing with labor costs.
Most AI budget requests are essentially one of three claims: replacing outsourced labor, replacing internal labor, or creating new revenue.
An employee has a salary. A BPO contract has a price per ticket, claim, invoice, or review. Humans understand these units. But inference cost is more complex because the final cost to complete a task depends on how the system runs during execution. A claims task that requires three retries, manual fixes, and calls to a frontier model might be more expensive than the outsourced labor it was meant to replace. That's why the conversation is turning to: What is the cost per outcome? Like per resolved ticket, per processed claim, per reviewed contract, per completed invoice, per job opening avoided, per customer retained, or per dollar of revenue converted.
Executives have realized BPO is the easiest place to establish a baseline because that work is already priced per 'unit completed.' Comparing internal employees to AI is much harder because an employee does many things in a day, including scrolling TikTok during lunch; productivity gains often show up as avoided hires or dispersed capacity release; and managers resist cutting headcount based on partial automation. BPO provides a quantifiable baseline for business teams.
This differs from the logic of SaaS. SaaS trained businesses to treat usage as a proxy for value.
AI breaks this. The same workflow can consume vastly different amounts of inference resources depending on the prompt, retrieved context, model selected, tools called, retry count, and whether the agent gets stuck. The unit on the bill—the token—is stable, but the workload it represents is not.
More accurately: signal and noise use the same unit of measurement. A rising token bill could mean real work is getting done; but it could also mean compute is being wasted on bad prompts, irrelevant context, unnecessary tool calls, repeated reasoning, and over-capable models. Two companies could have identical token bills, but the underlying operations are completely different: one is converting inference into results, the other is paying for wasted effort, and both look identical on the invoice lines.
SaaS usage tells you: the software has been adopted. AI usage can only tell you: the meter is running. It doesn't tell you whether the company is actually moving.
Why Is Marginal Token Utility Hard to See?
There are three main reasons.
First, the long tail of retries. If the probability an agent correctly completes a workflow on the first try is p, the expected token consumption per *solved* workflow roughly scales as T/p, where T is the base cost. If the completion rate drops from 90% to 70%, the effective cost per problem solved increases by about 28%, not 20%, because failures have a compounding effect. In enterprise workflows, inputs are often messy, and edge cases matter. Failure not only lowers accuracy but also changes the economics.
Second, context inflation. For operations heavily reliant on attention mechanisms, inference costs roughly scale O(n²) with context length. So, doubling context length roughly quadruples inference cost. Everyone wants the model to have enough information, so systems tend to oversupply: five documents would suffice, but retrieval fetches fifty; connectors dump entire email threads; agents carry stale conversation history forward.
Third, routing. When teams don't know which model is 'good enough,' the default is to use the most powerful one. A basic classification task might run on the same model meant for complex reasoning. At millions of calls, routing simple tasks to smaller models versus running everything on a frontier model is often the difference between a manageable bill and a board-level problem.
Non-software industries will feel this pain as a 'transformation.' Software companies will see it first because the work being optimized is already heavily instrumented. Engineering teams have metrics for PRs, commits, deploys, incidents, cycle time, MTTR, and these are linked to product. While not perfect, this work is easier to measure.
Non-software enterprises will feel this problem more profoundly because their work is operational. Like claims, underwriting, customer service tickets, compliance reviews, supply chain anomalies, payment disputes. Or, companies with real-world assets face the same issue. These workflows were traditionally measured in headcount, cycle time, SLA attainment, and error rate, often with higher requirements to stand up in an audit, not just be correct on average. The work unit and cost unit don't speak the same language or reside in the same organization. The tech team sees token consumption, the business sees workflow changes, but connecting the two requires multiple teams to first agree on 'what we're even measuring.'
I think software companies will experience the token budget war as a productivity measurement problem, aligning with many 'AI layoffs'; non-software enterprises will experience it as a transformation problem.
The missing layer is attribution from tokens to outcomes. Enterprises need a translation layer connecting inference spend to work completed and business results generated. This layer must answer three questions: What is the true cost of this workflow, including retries and fixes? In the agent's execution trace, which parts were essential, and which were wasted cycles? Did this work change the operating model—like fewer tickets per agent, shorter claims cycles, smaller BPO budgets, delayed hiring? The next level is outcome attribution in business language. Not just 'this workflow cost $2.13,' but: This type of claim is cheaper with an agent than BPO, but if the policy requires an extra exception document, the long tail of retries destroys the economics.
Measurement becomes memory. To connect a token to an outcome, a company must capture everything that happened in between: what the agent saw, what it retrieved, which tools it called, what it ignored, where it retried, when it was overridden by a human, which exception rule applied, which precedent mattered, and why one path succeeded while another failed. The measurement layer must log the decision trail, which is precisely what enterprises have almost never truly possessed. Logging systems capture what happened, but rarely why. A CRM can tell you a deal slipped, but not the unwritten judgment behind a sales forecast.
Reasoning behind decisions is one of the most perishable, most corruptible assets in a company because it lives in Slack threads, email chains, escalation meetings, and people's heads. The problem is, people leave, and processes change.
AI changes this because agents generate traces. Every retrieval, tool call, retry, escalation, human correction, and final decision becomes part of a path from context to action to outcome. Initially, companies capture these traces to justify spend. But once captured, these traces become more valuable than the cost report itself, as they turn into a durable record of how the organization actually makes decisions. (Ahem, context graph, though I'm quite tired of hearing that term lately.)
The allocation layer is the real prize. If inference becomes a metered resource in a company's operating model, then every dollar must justify itself. Which vendors can explain when tokens converted to outcomes, when they didn't, and why?
Enterprises won't figure this out entirely on their own. They'll buy it as a transformation. The Fortune 500 has seen this playbook before: buckle up, hire McKinsey, recruit every former Palantir employee on the market, and drive change top-down from the CEO. Token-to-outcome attribution will arrive similarly to ERP, BI, and digital transformation: as a 'program' with executive sponsorship, underpinned by infrastructure, eventually becoming the new source of truth. Founders who can do this will build different founding teams and be different archetypes themselves from traditional founders.
Whoever masters token-to-outcome attribution gets to make allocation decisions: which workflows deserve more compute, which should be capped, which should switch to cheaper models, which stay human, which can replace BPO. And once you can make those decisions, you control the flow of AI spend inside the enterprise and gain the trust needed to allocate that resource.
The first phase of enterprise AI proved: models can do work. The next phase will determine: how much of that work is worth paying for. As Charlie Munger said: Show me the incentive, and I'll show you the outcome.
Original Link
