Rolling Summary Policy

The problem, from first principles

Dropping old messages loses information. A gentler policy is a rolling summary: when the history grows past the budget, replace the oldest messages by a short summary of them, and keep recent messages verbatim because the model needs exact recent detail. The policy question is how many of the oldest messages to fold into the summary. Fold too few and you will have to summarize again next turn, fold too many and you throw away detail you could have kept. The best choice is the smallest number of old messages that brings the total, including the summary's own size, back under budget, never touching the most recent keep_recent messages.

From theory to code

Implement messages_to_summarize.

Constraints

• token_counts is a list with the size of each message, oldest first. budget is the maximum total. summary_tokens is the fixed size of the summary that will replace the folded messages. keep_recent messages at the end are protected.
• If sum(token_counts) <= budget, return 0 (nothing to do).
• Otherwise find the smallest k >= 1 with k <= len(token_counts) - keep_recent such that sum(token_counts[k:]) + summary_tokens <= budget and return it. If no such k exists, raise ValueError.

Hints