The new economics of AI cost management

The new economics of AI cost management | Databricks Blog Skip to main content

Summary

Enterprises are shifting from "tokenmaxxing" to "valuemaxxing," measuring the impact generated per dollar spent on AI.

Measuring AI ROI requires attribution granular enough to trace spend to a user, team, project or even a single pull request

Spend guardrails, end-to-end observability and a centralized point all AI traffic flows through are the foundation teams need now

For the past two years, the story around enterprise AI adoption has mostly been about getting people to use it. Over the past few months, that story has rapidly been changing in many organizations. AI observability that once measured success by token volume is starting to measure success by return on investment, and the enterprises that figure out cost management first are the ones that will be able to scale AI without their spend outrunning their revenue. At its core, AI cost management is the practice of measuring, attributing, controlling and optimizing AI spend across models, agents, applications and teams. The goal isn't simply to use fewer tokens, but to maximize the business value generated for every dollar spent. As AI agents take on more autonomous work, that discipline increasingly extends to managing the cost of agent reasoning, tool calls, and the models powering them. We sat down with Kevin Stumpf, a Senior Staff Engineer at Databricks and Technical Lead for Unity Gateway, to talk through how teams should actually put that discipline into practice: not treating AI cost as a line item to minimize, but as a system to manage. From tokenmaxxing to valuemaxxing Social media is full of leaders arguing that "tokenmaxxing," running AI as hard and as often as possible, is the right way to accelerate innovation. Other than the obvious cost implications, why is that the wrong way to think about AI adoption? Kevin Stumpf: The industry is moving from tokenmaxxing to valuemaxxing. In the early days, there were public leaderboards showing how many tokens people were burning through, and that made sense as a strategy at the time. A lot of people were reluctant to start using AI, so showing that the leaders everyone looked up to were heavy AI users gave others a reason to follow. If I see someone I respect using a ton of AI, I want to be like that person. But the system got gamed. Once being at the top of the leaderboard became a status symbol, people figured out the fastest way to get there wasn't good work, it was volume. Spin up a dumb agent that runs in a loop, have it ask what the weather is over and over, and burn through tokens doing essentially nothing. That was the predictable outcome of an incentive structure that rewarded raw usage instead of outcomes. Now that organizations are broadly bought into AI, the conversation has shifted to responsible, effective use. What we should really be building toward is a different kind of leaderboard, one that ranks people by the impact they generated with AI divided by what it cost to generate it. That's return on investment, and it's a much harder thing to quantify than raw token counts, but it's the number that actually matters. Related: What’s Driving Enterprise AI Adoption? What cost attribution really means in practice Can you walk us through what "cost attribution" really means in AI? Kevin Stumpf: AI cost attribution is ultimately about answering two questions: where does my spend come from, and what did I get for it? At the simplest level, that means attributing cost to an individual user or a team, like a data science team or a software engineering team. It can also map to a specific project, or to a node in an org chart, like understanding how much spend sits under a VP of AI versus an SVP of engineering. The goal is granular enough visibility that you can slice and dice the data and understand where cost is actually coming from, because you can't manage what you can't measure. Databricks provides cost attribution out of the box by user, group, agent , AI gateway endpoint, model, provider and time period. But organizations can also hyper-customize it using request-level tags. A tag can represent anything your organization cares about, like a specific pull request, a feature or an external customer, and once every request carries that tag, you can aggregate spend against it however you need to. That shift in customer sentiment has been real. Six months ago, the top concern was falling behind on AI adoption. Now it's the opposite: AI is often used for silly things, costs are growing faster than revenue, and leaders are asking how to bend that cost curve before it eats into profitability. This isn't a hypothetical. It's a boardroom-level conversation at nearly every organization we talk to, including internally at Databricks . Taming context bloat Token usage and context window size get blamed a lot, but are rarely explained well. What does context bloat actually look like in a production system, and what's the practical fix? Kevin Stumpf: Context bloat means throwing far more information at an agent than it needs to make a good decision. That's a problem for two reasons. First, it burns unnecessary tokens. Second, it's distracting: the agent has to process that extra information and decide whether it's relevant, which is its own kind of overhead. It's not unlike handing a person a stack of irrelevant documents and asking them to figure out what matters before they can even start the task. The best fix isn't trimming or caching after the fact, it's not generating the bloat in the first place. A common and effective pattern is letting the agent pull relevant context on demand, through tools, MCP servers or skills, rather than proactively pushing everything it might need up front. You should only push context to an agent by default if you're 90% certain or more that it's genuinely important for the task at hand. Otherwise, let the agent know what's available and allow it to fetch what it needs when it needs it. Dynamic model and harness routing Model selection is often a "pick once" decision. Why should it be dynamic instead, and how does intelligent routing decide, in real time, whether a task needs a frontier model or something cheaper and faster? Does that same idea extend to choosing between different agent harnesses? Kevin Stumpf: AI cost optimization isn't about...