AI Engineer Problems › Production AI

Chat History Trimming That Drops the System Prompt

Easy · Code Comprehension · Production AI

Scenario

A chatbot keeps the full conversation and sends it to the model each turn. To stay under the context limit, a helper trims the history to the most recent messages. After the change shipped, long conversations started behaving strangely: the bot forgot its persona and ignored its rules.

trim_history keeps only the last max_messages entries. The message list begins with a system message that sets the bot's rules, followed by alternating user and assistant turns.

Run the code as-is. After trimming, the first message is no longer the system prompt.

Your task: fix trim_history so the system message is always kept, while the rest of the history is still trimmed to the most recent turns. Keep the total number of messages at or below max_messages.

A real interview version of this asks: what else could go wrong when you truncate a conversation, and how would you measure it in tokens instead of messages?

Solve it in the browser on GenAIDojo