Why Your MCP Server Breaks Prompt Caching

If your agent loop pays full input price on every turn, the most likely cause is the tool block, not the conversation. Providers cache a literal prefix of the request, and MCP tool definitions are serialized ahead of the system prompt and the messages, so a single description built from live data shifts the bytes and invalidates everything behind it. The fix is to treat your serialized tool list as a build artifact that must be byte identical across turns, and to hash-test it in CI the same way you would a database migration.

Why tool definitions are the most expensive bytes you send

Tool definitions are the most expensive bytes in an agent request because they are re-sent in full on every single model call and they sit ahead of everything else in the cache hierarchy. A stateless Messages or Responses API call carries the entire tool array each time, so a server exposing 60 tools with an average of 250 tokens of JSON Schema each adds 15,000 input tokens per turn before the agent has said anything.

Anthropic documents the cache order explicitly as tools, then system, then messages, and states that a change to tool definitions invalidates the cached system prompt and message history as well (Anthropic prompt caching docs). OpenAI's automatic caching works the same way on a prefix basis, matching in 128 token increments above a 1,024 token floor, with tool definitions counted as part of that prefix (OpenAI prompt caching guide). The consequence is asymmetric in a way that surprises people: the tool block is the smallest part of the prompt you think about and the part with the largest blast radius when it changes.

MCP makes this easy to get wrong because the tool list is produced by a separate process over a protocol, at client startup and again on demand. The client calls tools/list, converts the result into provider tool format, and sends it. Your server decides those bytes, but you never see the request that carries them. If you want the wider background on caching mechanics, how to cut LLM API costs with prompt caching covers the provider side.

How to confirm you are actually missing the cache

Confirm a miss by logging the per-call usage counters, not by inferring it from a monthly bill. Every provider returns cache accounting on the response, and one session's worth of logs settles the question in minutes.

// Healthy: prefix written once, then read on every later turn
{
  "usage": {
    "input_tokens": 412,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 17034,
    "output_tokens": 96
  }
}

// Broken: the same 17k prefix is rewritten on turn 2, turn 3, turn 4...
{
  "usage": {
    "input_tokens": 412,
    "cache_creation_input_tokens": 17034,
    "cache_read_input_tokens": 0,
    "output_tokens": 96
  }
}

Log four numbers per model call, tagged with a session identifier: the two cache counters, the uncached input_tokens, and time to first token. Then compute cache read tokens divided by total prefix tokens across a session. A healthy agent loop lands above 0.85 after the first turn. Anything under 0.3 means the prefix is moving, and the next question is which bytes.

To find them, capture the serialized tool array on two consecutive turns and diff it. In Claude Code and similar clients you can get the same signal by dumping tools/list twice against a running server and diffing the JSON. If the diff is empty but the cache still misses, the problem is upstream of your server, in message rewriting or client-side ordering.

Five patterns that change your tool bytes between turns

Five things move tool bytes mid-session, and four of them live inside your MCP server. They are worth checking in this order because the first two account for most of what I have seen in production.

One, descriptions or schemas built from live data. A description that embeds the current date, an account name, a row count, or a list of valid project identifiers changes whenever that data changes. Enum values generated from a database table are the same bug with a nicer face: the schema is correct, and it is different on every deploy and sometimes on every request.

Two, nondeterministic serialization. Iterating a Python set to build an enum, discovering tool modules with os.listdir, or letting Pydantic emit $defs in whatever order it resolved them all produce schemas that are semantically identical and byte different across processes. This one survives code review because the JSON looks fine in isolation.

Three, per-user or per-role tool filtering that flips mid-session. If a permission check runs at tools/list time and the user's scope refreshes, the tool array shrinks or grows and the prefix is gone.

Four, notifications/tools/list_changed. The MCP specification lets a server declare the listChanged capability and notify clients that the tool list has changed, at which point a well behaved client re-fetches (MCP specification, Tools). Worth being precise here: the notification itself does not break the cache. Re-fetching an identical list is free. It breaks the cache only when the refetched bytes differ, which is exactly what the notification is for. Fire it on genuine capability changes, not on data changes.

Five, client-side ordering across servers. When a client concatenates tools from several MCP servers in connection completion order, a server that is slow to start on one run and fast on the next lands in a different position in the array. The bytes for your tools are stable and the prefix still moves. Some clients sort by name and some do not, so test rather than assume.

How to make tool definitions byte stable

Make them stable by generating the schemas once at process start, serializing with sorted keys and fixed separators, and never recomputing per request. Dynamic facts belong in tool results, where they sit at the end of the context and cost nothing in cache invalidation.

# Breaks the cache: the description embeds data that moves
@mcp.tool(
    description=f"Query orders. Valid projects: {', '.join(load_project_names())}. "
                f"Data current as of {datetime.now(timezone.utc).isoformat()}."
)
def query_orders(project: str, since: str) -> dict: ...

# Stable: constant description, live facts move into the result
@mcp.tool(
    description="Query orders for one project. Call list_projects first to "
                "resolve a project identifier. `since` is RFC 3339."
)
def query_orders(project: str, since: str) -> dict: ...

Replacing a generated enum with a companion list_projects tool costs one extra round trip on the first call of a session and saves the prefix on every call after it. That trade is worth taking for any tool whose valid values come from customer data. The same reasoning applies to the schema itself: pin it, and treat a change to it as a versioned event rather than a side effect, which is the argument in how to change an MCP tool without breaking agents.

Then fingerprint the output and test it. The official MCP Python SDK gives you the tool objects directly, so the test is short.

# tests/test_tool_stability.py
import hashlib, json

def fingerprint(tools):
    payload = json.dumps(tools, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(payload.encode()).hexdigest()

def test_tool_list_is_byte_stable(snapshot):
    tools = [t.model_dump(exclude_none=True) for t in list_tools()]
    assert fingerprint(tools) == snapshot  # fails on any schema drift

def test_tool_list_is_process_stable(run_in_fresh_process):
    # catches set iteration order, os.listdir order, $defs ordering
    a = run_in_fresh_process(lambda: fingerprint(dump_tools()))
    b = run_in_fresh_process(lambda: fingerprint(dump_tools()))
    assert a == b

The second test matters more than the first. Within one process, hash ordering is usually consistent; across two fresh interpreters it is not, and that is precisely the condition a client reconnecting to your server reproduces.

Where to put the cache breakpoint

Put one breakpoint at the end of the block that rarely changes, which for most agents means after the system prompt, so tools and system are cached together as a single stable prefix. Anthropic's API requires explicit cache_control markers, allows up to four breakpoints, and enforces a minimum cacheable length of 1,024 tokens for Sonnet and Opus class models with a higher floor for Haiku class models (Anthropic prompt caching docs). A 60 tool server clears that floor on the tool block alone.

{
  "model": "claude-sonnet-5",
  "tools": [
    { "name": "list_projects", "input_schema": { "...": "..." } },
    { "name": "query_orders",  "input_schema": { "...": "..." } }
  ],
  "system": [
    { "type": "text", "text": "You are...",
      "cache_control": { "type": "ephemeral" } }
  ],
  "messages": [ "..." ]
}

A marker on the last system block covers everything before it, tools included, so you do not need a separate marker on the last tool unless you want tools cached independently of a system prompt that changes per request. Add a second breakpoint near the tail of the message list once conversations run long, so growing history is cached incrementally rather than rewritten. The default entry lives for five minutes and refreshes on each hit, with a one hour option for loops that idle between turns.

OpenAI and Google handle this implicitly rather than with markers, which removes the placement decision and keeps the stability requirement identical: Gemini applies implicit caching to a matching prefix on 2.5 class models and offers explicit CachedContent for guaranteed behaviour (Gemini API context caching). No provider will cache a prefix that changes, whatever the API shape.

Why compaction invalidates more than the part you rewrote

Compaction invalidates everything after the edit point, because the cache matches a prefix and any change to an earlier message discards the whole tail. Truncating a 40,000 token tool result from turn 3 saves 40,000 tokens on the next turn and throws away the cached representation of turns 4 through 20, which you then pay to rewrite at 1.25x the base input rate.

That maths flips the usual instinct. Compact rarely, in large batches, and only at the oldest end of the conversation, so the rewritten prefix is amortised over many subsequent turns instead of being redone every few calls. Compacting on a fixed cadence, say every third turn, is close to the worst possible schedule: it is frequent enough to guarantee a cold prefix and not aggressive enough to keep the window small. Agent context compaction works through when to summarise against truncate; the caching cost belongs in that decision.

The better lever is upstream. If tool results are large enough that you are compacting to survive, cap them at the server, return a handle plus a preview, and let the agent request detail. Stopping MCP tool results from eating your context covers the shapes that work. A server that never emits a 40,000 token result never forces the rewrite in the first place.

What the miss costs on a representative loop

The cost is roughly 5x on the prefix portion of your input bill, and it hits latency before it hits the invoice. Take a server with 60 tools at an average 250 tokens of schema, so 15,000 tokens of tools, plus a 2,000 token system prompt, inside an agent task that makes 12 model calls.

Check current rates on the Anthropic pricing page, since the multipliers matter more than the absolute numbers here: a cache write costs 25 percent more than base input and a cache read costs 10 percent of it. That spread is what turns a small serialization bug into a five figure monthly line item.

Latency moves first and is what users notice. Reprocessing 17,000 tokens of prefix on every turn adds hundreds of milliseconds to time to first token, twelve times per task, on a loop where the agent is already making the user wait. Teams usually discover this as a vague complaint that the agent feels slow, and only find the cache miss later.

Trade-offs and what I would ship

The honest counterargument is that caching does not pay for short sessions. A cache write costs more than an uncached read, so a one-shot classification call that never revisits the prefix is cheaper without it, and a dynamic tool list is harmless there. The break-even is around two to three calls against the same prefix. If your workload is genuinely single-turn, stop reading and spend the effort elsewhere. If it is an agent loop, the prefix is read ten or more times and stability dominates every other consideration.

The second trade-off is discovery quality. Generated enums exist because they help the model pick valid arguments on the first attempt, and replacing them with a list_projects call costs a round trip and some chance the agent skips it. I still take the static schema, because a wrong first argument is recoverable through a well written error and a cold prefix is not recoverable at all. Write the error message to name the discovery tool explicitly, and the retry costs one cheap turn.

Third, dynamic tool loading. Tool search and code execution patterns cut the tool block dramatically, which is the right call once you are past a few dozen tools, but they change the tool array by design. The way to have both is to resolve the tool set once at session start and hold it for the session, rather than re-selecting per turn. Per-turn selection optimises the metric that matters least, since a 2,000 token tool block read from cache costs less than a 600 token block written fresh. That framing is worth holding alongside tool search against code execution.

What I would ship, in order: fingerprint the serialized tool list and fail CI on drift, move every live value out of descriptions and enums into results, log the two cache counters per model call with a session tag and alert when the read ratio drops below 0.5, place one breakpoint after the system prompt, and move compaction to a size threshold at the oldest end of the history instead of a turn cadence. The first two take an afternoon and account for most of the recovery.

Sources

• Anthropic, Prompt caching, cache hierarchy, invalidation rules, breakpoints, minimum lengths and TTL options.
• OpenAI, Prompt caching, prefix matching behaviour and the tokens included in a cached prefix.
• Google, Gemini API context caching, implicit and explicit caching.
• Model Context Protocol specification, Tools, tools/list and the listChanged capability.
• MCP Python SDK, tool registration and schema generation.
• Anthropic, Pricing, base input rates and cache write and read multipliers.