Token-Cost Aware Context Summarizer

Token-Cost Aware Context Summarizer

To control costs, RAG pipelines select relevant document chunks that fit into a budget constraint.

Implement summarize_context_cost_aware(chunks, max_tokens) where:

1. chunks is a list of dicts: {"text": str, "relevance_score": float}.
2. Each chunk's token usage is estimated as: ceil(word_count * 1.37) where word_count is the number of whitespace-separated strings in text.
3. Sort chunks by relevance score in descending order. If there is a tie, sort by token count in ascending order. If there is still a tie, preserve their original relative ordering.
4. Greedily select chunks from the sorted list if adding them does not exceed max_tokens.
5. Return the combined text of all selected chunks joined by spaces, ordered by their original input sequence index.

Example

chunks = [
    {"text": "high relevance text", "relevance_score": 0.9}, # words=3, tokens = ceil(3 * 1.37) = 5
    {"text": "low relevance text that is longer", "relevance_score": 0.4}, # words=6, tokens = ceil(6 * 1.37) = 9
    {"text": "medium relevance text", "relevance_score": 0.7} # words=3, tokens = ceil(3 * 1.37) = 5
]
summarize_context_cost_aware(chunks, 11)
# Returns: "high relevance text medium relevance text" (both have 5 tokens, total 10 <= 11)