MCP Caching: Optimizing ttlMs and cacheScope for Agents
Source: mortalapps.com- MCP caching allows servers to define the validity and visibility of tool responses using ttlMs and cacheScope metadata.
- It solves the problem of redundant backend API calls during iterative agent reasoning loops.
- Production significance lies in reducing P99 latency and preventing thundering herd scenarios on legacy systems.
- Correct implementation ensures agents operate on fresh data while minimizing operational costs and token consumption.
Why This Matters
In production agentic systems, the 'reasoning loop' often involves an agent calling the same tool multiple times to verify state or retrieve incremental updates. Without a standardized caching mechanism, every tool execution triggers a full round-trip to the backend infrastructure. This approach was invented to decouple the high-frequency polling behavior of LLMs from the underlying data sources, which are often rate-limited or computationally expensive to query. If you ignore MCP caching, your system will likely suffer from 'agentic thundering herds' where a single user request results in dozens of identical, expensive API calls, leading to unnecessary latency and potential service outages. From an enterprise perspective, caching is not just a performance optimization but a cost-control and reliability requirement. It allows developers to specify exactly how long a piece of information remains valid (ttlMs) and who is allowed to see it (cacheScope). This is superior to client-side caching because the server - the domain authority - dictates the freshness policy based on the volatility of the underlying data. Use this when your tools wrap external APIs, databases, or long-running computations where the result is idempotent for a known duration. Avoid this for tools that perform state-changing actions, such as 'delete_record' or 'send_email', where caching would lead to incorrect agent perceptions of the system state.
Core Concepts
The Model Context Protocol introduces specific metadata fields to manage the lifecycle of tool outputs. Understanding these terms is foundational to building efficient agentic connectors.
- ttlMs (Time-to-Live in Milliseconds): A numeric value indicating the duration for which a client should consider a tool response valid. Once expired, the client must re-execute the tool.
- cacheScope: A string enum defining the visibility boundary of the cached result. It determines if a result can be shared across different sessions or users.
- Global Scope: Indicates the response is generic and can be served to any client requesting the tool with identical arguments (e.g., a public weather report).
- Session Scope: Restricts the cache to the current connection or user session, preventing sensitive data leakage (e.g., a user's private inbox status).
- Idempotency: The property where multiple identical requests return the same result. Caching is only safe for idempotent tool operations.
- Argument Hashing: The process by which a client generates a unique key for a cache entry based on the tool name and the specific input parameters provided by the LLM.
How It Works
The MCP caching lifecycle is a coordinated effort between the server and the client host. It follows a structured sequence to ensure data integrity.
1. The Tool Request Phase
When an agent decides to use a tool, the client host first checks its local cache. It generates a hash of the tool name and the JSON-serialized arguments. If a match is found and the ttlMs has not elapsed, the client returns the cached result immediately, bypassing the server entirely.
2. Server Execution and Metadata Attachment
If a cache miss occurs, the request is forwarded to the MCP server. The server executes the underlying logic (e.g., querying a database). Along with the standard content array, the server attaches a _meta object containing the ttlMs and cacheScope. This metadata informs the client how to handle the result for future requests.
3. Client-Side Storage
Upon receiving the response, the client stores the result in its cache. The storage logic respects the cacheScope. If the scope is session, the entry is tagged with the session ID. If global, it is stored in a shared pool. The client also calculates the absolute expiration time by adding ttlMs to the current system clock.
4. Failure and Invalidation
If the backend API fails during execution, the server should not return a cacheable metadata object, or it should set ttlMs to 0. Clients must also implement a mechanism to invalidate the cache if an agent performs a related state-changing action that would render the cached data stale, though this is often handled by short TTLs in dynamic environments.
Architecture
The architecture consists of four primary layers. Execution begins at the Agent Runtime (Client).
- Agent Runtime: Hosts the LLM and the MCP Client. It contains the Cache Manager, which intercepts tool calls.
- Cache Store: A key-value store (in-memory for local agents, Redis for distributed systems) that maps argument hashes to tool outputs and metadata.
- MCP Server: The middleware that translates protocol requests into backend actions. It is responsible for calculating the appropriate TTL based on data volatility.
- Backend Systems: The source of truth (APIs, DBs). Data flows from the Backend to the Server, where metadata is appended, then to the Client, where it is stored in the Cache Store before being returned to the Agent.
The Decision Framework for ttlMs
Selecting the correct ttlMs requires balancing agent performance against data accuracy. For static or slow-changing data, such as organizational charts or documentation, a TTL of several hours (e.g., 3,600,000 ms) is appropriate. For highly dynamic data like system metrics or stock prices, a TTL should be measured in seconds (e.g., 5,000 ms to 30,000 ms). A common production pattern is 'Adaptive TTL,' where the server adjusts the duration based on the current load or the specific resource being accessed. If a backend is under heavy load, the server might temporarily increase ttlMs to protect the resource.
Scoping with cacheScope: Global vs. Session
The cacheScope parameter is the primary mechanism for ensuring multi-tenant security.
- Global Scope: Use this for data that is invariant across all users. Examples include public API documentation, general knowledge bases, or shared configuration files. Global caching maximizes hit rates across the entire system.
- Session Scope: Use this for any data that is user-specific or context-dependent. If a tool retrieves 'My Recent Tasks,' it must be scoped to the session. Failure to use session scoping for private data is a critical security vulnerability that can lead to cross-user data leakage.
Cache Key Generation and Collisions
Clients generate cache keys by hashing the tool name and its arguments. In production, it is vital to use a deterministic serialization method (like sorted JSON keys) before hashing to ensure that {"a": 1, "b": 2} and {"b": 2, "a": 1} produce the same key. Using a cryptographic hash like SHA-256 prevents collisions. Developers must also ensure that the cacheScope is part of the key derivation logic to prevent a global cache entry from being overwritten by a session-specific one.
Handling Non-Deterministic Tools
Some tools are inherently non-deterministic, such as those generating random numbers or timestamps. These tools should never return a ttlMs greater than 0. If a tool's output changes every time it is called regardless of input, caching will break the agent's logic. The server developer must explicitly omit caching metadata for these tools to ensure the client always fetches a fresh result.
Code Example
from mcp.server.fastmcp import FastMCP
import time
import os
mcp_server = FastMCP("weather-service")
@mcp_server.tool()
async def get_current_weather(city: str) -> dict:
# In a real scenario, this hits a paid API
# We use ttlMs to reduce API costs
weather_data = {"temp": 22, "condition": "Cloudy"}
return {
"content": [
{"type": "text", "text": f"The weather in {city} is {weather_data['temp']}C and {weather_data['condition']}."}
],
"_meta": {
"ttlMs": 600000, # Cache for 10 minutes
"cacheScope": "global"
}
}
if __name__ == "__main__":
mcp_mcp_server.run()A JSON response containing the weather text and a _meta object with ttlMs: 600000 and cacheScope: 'global'.
@mcp_server.tool()
async def get_user_preferences(user_id: str) -> dict:
# Sensitive data must be session-scoped
user_prefs = {"theme": "dark", "notifications": True}
return {
"content": [
{"type": "text", "text": f"Preferences for {user_id}: {user_prefs}"}
],
"_meta": {
"ttlMs": 300000, # 5 minutes
"cacheScope": "session"
}
}A JSON response where the _meta object specifies 'session' scope, ensuring the data isn't shared with other users.