Fixing MCP Tool Bloat: Tool Search vs Code Execution

Anthropic measured its own internal agent setups carrying 134K tokens of tool definitions before any work began, and puts the point where Claude's tool-selection accuracy starts slipping at 30 to 50 available tools. Those two numbers, not model capability, are what cap most production agents that aggregate several MCP servers. The remedy is architectural: stop shipping the entire tool catalog on every request. Two patterns now compete for that job, on-demand tool search and code execution, and they solve different halves of the problem.

Key takeaways

Anthropic clocks a five-server setup (GitHub, Slack, Sentry, Grafana, Splunk) at roughly 55K tokens of tool definitions before the first user message, and reports 134K tokens in its own internal configurations before optimization.
Selection accuracy usually breaks before the context window does. Anthropic places the degradation point at 30 to 50 tools; the RAG-MCP paper measured baseline tool-selection accuracy at 13.62% versus 43.13% with a retrieval layer in front of the catalog.
Deferred loading via the tool search tool cut a 77K-token tool preamble to about 8.7K, an 85% reduction, and lifted Claude Opus 4 from 49% to 74% on Anthropic's internal MCP evaluation.
Code execution attacks tool results rather than definitions. Anthropic reported one workflow dropping from 150,000 tokens to 2,000; Cloudflare reported 1.17 million tokens down to roughly 1,000 for its 2,500-endpoint API.
Neither pattern is free. On the tau-squared benchmark, where each turn makes only one or two sequential calls, programmatic tool calling left task scores unchanged and cost about 8% more.
The 2026-07-28 MCP specification adds a server/discover method plus ttlMs and cacheScope metadata on tools/list, pulling discovery and caching into the protocol itself.

What Does Tool Bloat Actually Cost an Agent?

It costs you twice: once in context you paid for and never used, and once in accuracy you cannot buy back with a bigger model. Anthropic's own figure for a plain multiserver setup, GitHub plus Slack plus Sentry plus Grafana plus Splunk, is about 55K tokens of definitions consumed before the conversation starts. Internally the company has seen 134K.

The first cost is easy to see on an invoice. Every request re-sends those definitions, and every cached-miss turn pays full price for schema text the model will ignore. A 200-tool aggregation layer can spend more input tokens describing capability than exercising it.

The second cost is the one that ends projects. Tool definitions are not inert padding; they are candidates the model must discriminate between on every decision. Speakeasy built a deliberately pathological case, an OpenAPI document with 107 near-identical GET operations turned into 107 MCP tools, and found that at 20 tools a small model got 19 of 20 calls right, while at 107 tools both large and small models failed to select correctly and started hallucinating tool names.

Senior engineers should read that as a design constraint rather than a model defect. The request you send is the search space you asked the model to solve, and you control its size.

Why Does Accuracy Fall Before the Context Window Fills?

Because selection is a discrimination problem, and adding tools adds confusable neighbours faster than it adds capability. Fifty tools named get_user, get_user_profile, fetch_user_record and lookup_customer do not give the model fifty options; they give it one decision with fifty overlapping descriptions, most of which were written by different teams with different naming conventions.

The retrieval literature quantifies the gap. In RAG-MCP, Gan and Sun measured tool-selection accuracy at 13.62% when the full catalog sat in the prompt, against 43.13% when a retrieval step narrowed the candidates first, roughly a threefold improvement with no change to the underlying model. Anthropic's internal MCP evaluation shows the same shape from the other direction: Opus 4 moved from 49% to 74% and Opus 4.5 from 79.5% to 88.1% once tool search replaced the full preamble.

Model size buys you headroom, not immunity. The effect is sharpest on smaller and locally hosted models, which is why teams doing tool calling against a local model in Ollama tend to hit the ceiling at a dozen tools rather than fifty. This is also a clean example of the broader point that the agent harness rather than the model decides observed performance: the same weights score twenty-five points apart depending on how you shaped the request.

How Does Tool Search Keep Definitions Out of Context?

By separating what you send from what the model sees. With Anthropic's tool search tool you still transmit every tool definition in the tools array on every request, but any tool marked defer_loading: true is excluded from the system-prompt prefix. Claude sees only the search tool and whatever you left non-deferred, then queries the catalog when it needs something.

The API returns matches as tool_reference blocks, up to five per search by default, and expands them into full definitions inline before Claude reads them. Two variants ship: a regex variant where Claude writes Python re.search() patterns capped at 200 characters, and a BM25 variant that takes natural-language queries up to 500 characters. Both search tool names, descriptions, argument names, and argument descriptions, which is why namespacing matters more than prose quality here. Prefix by service, github_ and slack_, and a single pattern pulls the whole group.

The reported saving on a realistic setup is 77K tokens down to about 8.7K, an 85% reduction that preserves roughly 95% of the context window. The design detail worth noting is what it does not break: because deferred definitions never enter the prefix and references are appended inline, prompt caching survives, and strict-mode grammars still compile from the full toolset. Teams whose cost model already leans on cached prefixes keep that saving intact.

Constraints to design around: at least one tool must stay non-deferred (normally the search tool itself, and a 400 error tells you if you deferred everything), a deferred tool cannot also carry cache_control, the ceiling is 10,000 deferred tools per request, and MCP connector tools are configured at the mcp_toolset level via default_config rather than per definition.

When Does Code Execution Beat Tool Search?

When your problem is the size of tool results rather than the size of tool definitions. Tool search shrinks the preamble; it does nothing about a 10,000-row spreadsheet or a two-hour transcript passing through context twice on its way from one tool to another.

Code execution changes the calling convention. Instead of one model round trip per tool call, the model writes a short script, the runtime executes it against the tool surface, and only the script's output returns to context. Anthropic's write-up on this pattern reports a workflow falling from 150,000 tokens to 2,000, a 98.7% saving, largely because intermediate data stops travelling through the model. Cloudflare's Code Mode makes the same bet at API scale, exposing over 2,500 endpoints through two functions, search() and execute(), and reporting roughly 1,000 tokens of context where the conventional approach would need 1.17 million.

Anthropic's managed version, programmatic tool calling, publishes the numbers with the caveats attached, which is what makes them useful. On a 75-tool project-management agent benchmark it cut billed input tokens by about 38% with no change in task accuracy. On complex research tasks average usage fell from 43,588 to 27,297 tokens, a 37% reduction. Added on top of basic search tools on BrowseComp and DeepSearchQA, it improved performance by an average of 11% while using 24% fewer input tokens. Across production traffic with 10 to 49 tool definitions, typical savings run 20%-40%.

Then the counterexample. On the tau-squared benchmark, covering airline, retail, and telecom domains where each turn makes one or two sequential calls, programmatic tool calling left scores unchanged and cost roughly 8% more. Container startup and script generation are fixed overhead; strictly sequential workflows where the model must reason over each result before the next call cannot amortise it. Fan-out across many items, large results you want filtered before they land, and iterative retrieval are where the trade pays. That is the same accounting discipline behind deciding whether to summarize or truncate agent context: the question is never whether a technique saves tokens in the abstract, but whether it saves them on your traffic shape.

Which Pattern Fits Which Workload?

Five approaches are in production use, and they are not substitutes so much as layers.

Approach	Mechanism	Reported token effect	Accuracy effect	Where it breaks
Static full toolset	All definitions in every request	Baseline: ~55K tokens for five servers, 134K observed internally at Anthropic	Degrades past 30 to 50 tools	Any aggregation of more than a handful of servers
Curated toolset or gateway filter	Humans or a proxy expose only the tools a workflow needs	Proportional to what you cut	Best per-token accuracy when the curation is right	Brittle as workflows change; someone must own the allowlist
Tool search (deferred loading)	Definitions withheld from the prefix, discovered on demand	77K to ~8.7K, an 85% reduction	Opus 4 from 49% to 74% on Anthropic's MCP eval	Poorly named or thinly described tools become undiscoverable
MCP-native dynamic toolsets	Server exposes toolsets the client loads as needed, no sandbox required	Speakeasy reports ~96% input-token reduction across 40 to 400 tools	100% task success in their benchmark at every toolset size	2x-3x more tool calls and about 50% longer execution time
Code execution (Code Mode, programmatic tool calling)	Model writes a script; only its output enters context	150,000 to 2,000 tokens reported; 1.17M to ~1,000 at Cloudflare	Unchanged to +11% depending on workload	Sequential single-call turns; costs ~8% more on tau-squared

Read the last column first. Every row after the baseline trades a known, bounded operational cost for token and accuracy headroom, and the right answer depends on which cost your team can actually absorb.

What Breaks When You Turn These On?

Mostly the assumptions you did not know you were making. The documented sharp edges are worth reading before a migration, not after.

Scoping is guidance, not enforcement. Anthropic's docs state plainly that allowed_callers shapes how a tool is presented to Claude and is validated against tool_choice, but it is not a hard API-level block and should not be treated as a security boundary. Your client must still handle a direct tool_use for any tool it defines.
Tool results become untrusted input to an interpreter. Results return as strings into an execution environment, so a tool that relays external content creates a code-injection path. Cloudflare's answer is a Dynamic Worker isolate with no filesystem, no environment variables to leak, and external fetches disabled by default. If you self-host the runtime, the isolation tier you pick is a real decision, and the trade-offs between containers, gVisor, and microVMs for sandboxing agents apply directly.
Feature interactions bite. Programmatic calling does not support tools with strict: true, cannot be forced through tool_choice, rejects disable_parallel_tool_use, and returns a 400 with Circular $ref detected for recursive schemas that direct calling accepts. MCP connector tools cannot be called programmatically at all.
Platform coverage is uneven. Programmatic tool calling runs on the Claude API, Claude Platform on AWS, and Microsoft Foundry with a Hosted on Anthropic deployment, and is not available on Amazon Bedrock or Google Cloud. Server-side tool search on Bedrock works only through InvokeModel, not Converse.
Operational details. A pending programmatic tool call times out after about four minutes and raises a TimeoutError inside the script. Container artifacts and outputs are retained for up to 30 days, which your data-governance review will ask about.
Latency moves in the wrong direction. Speakeasy's dynamic-toolset benchmark traded 96% fewer input tokens for 2x-3x more tool calls and roughly 50% longer execution, typically six to eight calls where a static toolset made three.

How Should You Sequence This Work?

Measure first, because three of the five approaches above can make your specific workload worse, and the only way to know is your own traffic. Anthropic's guidance is explicit on this point: measure billed input tokens with and without the feature on a representative sample before enabling it broadly.

Count what you are actually sending. Sum the token cost of your tool definitions and compare it to the tokens spent on real work. If definitions exceed 10K tokens, or you have crossed ten tools, you are in the zone where deferral pays.
Prune and rename before you optimise. A curated 13-tool surface beats a cleverly deferred 200-tool surface for most single-purpose agents. Consistent namespacing and keyword-rich descriptions also determine whether search can find anything, so this step is a prerequisite rather than an alternative. If you own the server, the naming conventions you set when you build an MCP server in Python with FastMCP are what downstream search quality depends on.
Defer the long tail. Keep the three to five tools used on nearly every request non-deferred, mark the rest defer_loading: true, and put your cache breakpoint on a non-deferred tool.
Add code execution only where the shape fits. Fan-out over many records, large results needing filtering, and agentic retrieval. Leave strictly sequential, reason-between-calls workflows on direct tool use.
Re-run an eval, not a demo. Both patterns change failure modes as much as they change cost. A tool the model can no longer discover is a silent capability regression that a token-usage dashboard will never surface.

What to Watch in the Protocol

The workarounds are being absorbed into the specification. The MCP release published on 2026-07-28, with its release candidate locked on 21 May 2026, adds a server/discover method so clients can fetch capabilities upfront, and attaches ttlMs and cacheScope metadata to tools/list responses so clients can cache tool catalogs deliberately instead of refetching them.

It also drops session management for a stateless architecture, moving client info and capabilities into _meta on every request, and introduces a negotiated extensions framework with two official extensions at launch, MCP Apps for server-rendered UIs and Tasks for long-running work.

Read together, those changes say the ecosystem accepts that clients must be able to interrogate and cache a tool catalog rather than swallow it. Gateway-side filtering and bespoke tool-search layers are useful today and will look like transitional infrastructure within a year. The durable investment is the measurement harness that tells you which tools your agents actually call, and how often they pick the wrong one.