Reducing AI-agent costs: why 20 separate agents cost more than one central layer
Reducing AI-agent costs is rarely about a more expensive or cheaper AI model. It's about how many times the same data gets fetched. A company that builds twenty agents and lets each one build its own connection to the CRM, the ERP, or the inventory system pays twenty times for information that was already there. Gartner predicts the average Fortune 500 company will be running more than 150,000 AI agents by 2028, up from fewer than 15 in 2025. Almost nobody has a cost picture attached to that.
Episode 1 Deploying autonomous AI-agents safely named the hype around autonomous agents. Episode 2 Managing AI-agent access rights introduced the stoplight mechanism for keeping access and permission separate. This episode shows what it costs when that mechanism isn't run centrally anywhere.
What reducing AI-agent costs actually means in practice
At most companies, the cost problem doesn't start on purpose. A marketing team builds an agent that summarizes campaign data. A finance team builds one that checks invoices. An operations team builds one that checks inventory levels. Each of those decisions makes sense on its own.
The problem is that none of those agents knows the other two exist. Research firm Kore.ai describes a recognizable pattern: when five teams independently build a similar agent, say a document summarizer, the combined redundancy adds up to 20-35% of an organization's total AI budget. Five separate contracts, five times the same infrastructure, five sets of API tokens, for work one agent with the right access could just as easily have done for everyone.
At a transport company with around eighty people on the payroll, that looks like this: a planning agent, a billing agent, and a customer service agent all independently request the same trip data from the same system. Three logins, the same data sent over the line three times, three separate bits of maintenance every time that system gets an update. Reducing AI agent costs starts with asking whether that needs to happen once or three times.
How many AI-agents does a company actually have, and what does not knowing that cost
Gartner calls it agent sprawl: the uncontrolled proliferation of AI-agents without central oversight. According to their press release from late April 2026, the average Fortune 500 company had fewer than 15 agents in 2025. By 2028 that number is projected to exceed 150,000, a ten-thousand-fold increase in three years. Only 13% of organizations believe their own governance can keep up, and only 18% have a complete, current list of which agents are running.
Research from OutSystems among nearly 1,900 IT leaders worldwide, published in April 2026, confirms the same gap from a different angle: 94% see agent sprawl as a direct cause of more complexity, more technical debt, and more security risk. Only 12% have a central platform set up to manage it. Deloitte's State of AI in the Enterprise 2026, based on more than 3,000 senior leaders in 24 countries, sees agent usage climbing from 23% today to an expected 74% within two years, while only 21% have a mature governance model in place.
Amazon saw the same problem grow large enough to launch its own product for it in April 2026: Agent Registry, a central catalog of every agent within an organization. AWS described the problem itself well: once companies scale to hundreds or thousands of agents, keeping track of what's running, owned by whom, and whether it's approved becomes "an operational crisis" in its own right. That's not an abstract concern. It's precisely the moment reducing AI agent costs stops being an optimization and becomes a necessity.
Why the bill climbs so fast: the token side of AI agent costs
Alongside duplicate connections, there's a second, less visible cost factor: how an agent itself handles tokens. An AI agent doesn't work like a chatbot that gets asked one question and gives one answer. An agent works through a series of steps, and at every step it resends the entire accumulated history: the system prompt, earlier tool calls, earlier answers, plus the new step.
Research firm LeanOps analyzed thirty engineering teams running agents in production and found that AI-agents consume 10 to 100 times as many tokens as a chatbot for the same task as a result. Across those thirty teams, an average of 62% of the bill turned out to be exactly that: the same, unchanged context being resent over and over.
The same research describes a SaaS company with 35 engineers that cut its monthly agent costs from $87,000 to $24,000 after an audit. No model was replaced. The causes were recognizable one by one: no caching of the system prompt, a review agent defaulting to the most expensive model for routine work, and no budget cap per user at all. After three weeks of adjustments: a $756,000 annual saving, with no drop in productivity.
That's the same logic as the transport company case above, just within a single agent instead of between agents. Reducing AI-agent costs rarely comes down to choosing or avoiding a more expensive model. It comes down to stopping paying twice for the same thing, whether that's two agents fetching the same data or one agent resending the same context twenty times over.
What legislation still doesn't solve for reducing AI-agent costs
Since August 2, 2026, the EU AI Act has also been enforcing Article 15, which requires cyber resilience for what the law calls the action layer: the APIs and systems an agent actually calls, not just the model itself. Article 12 additionally requires logging of those interactions. Neither article lowers costs. They do the opposite: every extra, separately built agent isn't just an extra cost item, it's also an extra piece of action layer that has to be logged and secured.
Twenty uncoordinated agents that all independently reach the same data aren't just twenty times as expensive. They also produce twenty times as many separate pieces of action layer, all of which have to be documented the moment a regulator asks about it. That topic gets its own, full episode later in this series. For now, the core point is enough: reducing AI-agent costs and limiting compliance burden run largely along the same path.
How to actually bring AI-agent costs down structurally
We're not against agents, quite the opposite. Build as many as a department needs. The problem is never the number of agents. It's how many of those agents each build their own separate route to the same data.
1. One central retrieval layer instead of twenty separate connections
Instead of twenty agents all logging into the CRM, the ERP, and the financial records themselves, a handful of agents can get by with exactly one connection. To a layer that retrieves, validates, and passes on centrally. Fewer tokens, fewer separate connections, fewer places where something can go wrong.
2. Know how many AI-agents are running before it becomes a cost problem
Gartner's own recommendation starts with an inventory, not a ban. AWS even built a separate product for it. Without a list of which agents exist and what they touch, you only discover redundancy once the bill arrives.
3. Validate once, not twenty times
This is exactly where we've been for years, long before "agent" showed up on every slide. We validate what moves between systems at the moment it moves, once, centrally. For an agent requesting customer data, it's the same approach: validate at the source, not at every separate agent that would otherwise walk there on its own.
4. Hold costs accountable to results, not enthusiasm
LeanOps draws a hard conclusion from their own thirty audits: without deliberate cost management, agent costs scale with the builder's enthusiasm, not with the value delivered. An agent nobody actively monitors keeps costing until someone happens to notice.
Twenty agents all reaching the same data feels like progress, because each agent solves its own problem. The bill says something else.
How reducing AI-agent costs actually works at Compass
At Compass, we put in place exactly the layer missing from the four steps above. One central connection between all your agents and the systems they need something from, such as the CRM, ERP, financial records, or inventory system. Instead of every agent logging in itself, maintaining its own connection, and fetching the same information itself, that layer does it once: retrieve, validate against the rules that apply, and pass it on to whoever asks.
That saves money in three places at once. One connection instead of twenty means one piece of maintenance instead of twenty, including the moment a source system gets an update. One validation point instead of twenty separately built checks means less chance an agent misreads something and has to redo the work, along with all the tokens and time lost doing so. And because that layer already records what passes through it, the logging Article 12 of the AI Act requires comes with it automatically, instead of someone having to reconstruct it per agent afterward.
This isn't a future plan. We already do this for connections without an agent on the other end, and we're extending it now to agents as a new kind of requester of that same information: same validation layer, same discipline, new user.
Count, for your own company, how many agents currently reach the same CRM, ERP, or inventory system separately. Do you know that number off the top of your head, or would you have to ask three different teams before you had an answer?
Frequently Asked Questions
It varies a lot. An agent that only fetches and summarizes data costs little. An agent that works through many steps and resends the entire history at every step can consume ten to a hundred times as many tokens for the same end result, simply because of how it's built.
Usually something else. Choosing the most expensive model is rarely the main cause. The main cause is duplicate work: multiple agents fetching the same data separately, or one agent resending the same context over and over without anyone tracking it.
No. The number of agents isn't the problem. Twenty agents with one shared, validated path to data cost less than three agents that each build their own separate connection. The question isn't how many agents, but how many separate routes to the same data.
At most companies, nobody knows exactly. Gartner found that only 18% of organizations have a complete, current inventory. Starting with a simple list, who built which agent and which systems does it touch, often already exposes the biggest overlap.
Both. The same layer that stops twenty agents from fetching the same data separately is also the layer that tracks who was allowed to carry out which action. Reducing costs and keeping oversight run along largely the same track here.
Start with the agents that touch the same systems, not with every agent at once. If two or more agents reach the same CRM or ERP system, that's the first place a single shared connection saves money immediately.
Yes. This is exactly what we already do: fetch data centrally, validate it, and pass it on, so neither an agent nor a system has to build its own separate route to the same source. For AI agents, it's the same approach applied to a new kind of requester. No future plan, we set this up as a project, tailored to your systems and your agents.
