AI Engineer Problems › Production AI

Retry Logic That Burns Your Token Budget

Easy · Code Comprehension · Production AI

Scenario

Your service wraps an LLM call in a retry helper so transient failures don't surface to users. After a provider outage last week, finance noticed the bill spiked far beyond what the extra traffic explained.

call_llm is a mock of the model call. It fails with a retryable error twice, then succeeds, and it tracks how many "tokens" were spent across all attempts. call_with_retry is the wrapper someone wrote.

Run the code as-is and look at the final line. The request succeeded on the third try, but the token count is far higher than three attempts should cost.

Your task: find why retries are so expensive and fix call_with_retry. Do not change call_llm. It is a mock of the provider and behaves correctly.

Think about it the way an interviewer would ask it: what is actually being resent on each retry, and does it need to be?

Solve it in the browser on GenAIDojo