Context Window Overload in Multi-Item Shopping Agent
An e-commerce assistant agent helps customers find items in their order history ("Find all tech items under $50 shipped to Chicago in 2024"). A naive implementation fetches all past orders from the database and inserts them into the prompt. However, users with large order history (hundreds of items) will overwhelm the context window, causing high token costs, retrieval dilution, and slow response times.
How would you architect a search and query-planning engine to prevent context window overload while preserving accuracy?