Scenario
You own an MCP server that gives an LLM agent a get_weather tool. The upstream weather API is paid and rate-limited to 3 requests per minute, so last week's v2 release added a response cache: once a city has been looked up, repeat questions are answered from the cache instead of spending quota.
Since the release, users keep hitting this:
"I asked about Denver a couple of times, then London, and the assistant said the weather service was rate limited."
The cache is clearly there, yet the quota still runs out after asking about only two cities. Nothing crashes.
The code
WeatherAPIis a stand-in for the paid upstream API.UPSTREAM.callscounts paid requests, and it refuses anything past the limit.get_weather(city, units)is the tool itself.TOOLSis what the server advertises intools/list. The model reads each tool'sdescriptionandinputSchema(a JSON Schema) to decide how to call it.cache_key()and_cacheare the v2 response cache.handle_tools_call(request)handles an MCPtools/callrequest (JSON-RPC 2.0). It checks required arguments, looks in the cache, runs the tool, and wraps the output as{"content": [...], "isError": ...}.
Press Run to replay a conversation. Notice how the agent writes the city each time it calls the tool: LLMs rarely send the same string twice.
Your task
Find why the cache isn't saving quota, and fix it:
- Requests for the same city must share one upstream call, however the agent capitalizes or spaces it (
"Denver","denver","DENVER "). - Requests that would get a different answer, such as different
units, must not share a cache entry. - Keep the cache and the rate limit. Removing the cache or raising the limit isn't a fix. The quota is real money.
- Keep the MCP behavior as it is: unknown tools and missing required arguments are JSON-RPC errors, and a failing tool (like an unknown city) returns a normal result with
isError: true.
Use the Explain tab to write the root cause in one or two sentences, as you would say it to an interviewer.
Examples
ask {"city": "Denver"} -> Denver, Sunny, 20 celsius
ask {"city": "denver"} -> Denver, Sunny, 20 celsius (from cache)
ask {"city": "DENVER "} -> Denver, Sunny, 20 celsius (from cache)
ask {"city": "London"} -> London, Rainy, 9 celsius (today: "rate limit exceeded")
upstream calls: 2
ask {"city": "Denver", "units": "fahrenheit"} -> 68 fahrenheit, a separate cache entry
Things to think about
- When does the cache decide two requests are "the same"? What exactly does it compare?
- The tool already copes with
"DENVER ". Where does it do that, and what does the cache do differently? - What would happen to the quota if 1,000 users asked about the same city, each phrased slightly differently?