Skip to content

fix(core): shorten compaction transcripts oldest first before dropping history - #58015

Merged
rekram1-node merged 4 commits into
devfrom
staged-summary-shrink
Oct 22, 2026
Merged

rekram1-node merged 4 commits into
devfrom
staged-summary-shrink

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

When a compaction request is too big to send unchanged, history gets turned into a text transcript. That step did everything at once: it kept every tool definition, cut every tool output to 1,250 characters, and if that still didn't fit, dropped whole exchanges from the oldest end. Dropped exchanges end up in neither the summary nor the kept recent messages, so the summary loses them entirely, user requests included, while newer exchanges still carry long tool output.

This hurts most when a long session moves to a model with a much smaller window, and in sessions with a lot of MCP tools, where tool definitions are a big chunk of the request the summary doesn't use.

Separately, the kept recent messages written into the checkpoint included [Assistant reasoning]: ... lines. That text is the provider's summary of its reasoning, it inflates the checkpoint (#51818), and written out as text it can read like an attempt to extract the model's thinking.

Change

The transcript (system prompt kept, no tool definitions, no reasoning, no images) is shortened one exchange at a time, oldest first, through these levels, and stops as soon as it fits:

level 0  whole
level 1  tool outputs cut to 2,000 chars (start + end), inputs whole
level 2  tool outputs left out, tool inputs cut to 2,000 chars
level 3  only the user's text, cut to 2,000 chars
level 4  omitted ("[N older exchanges omitted]")

for level 1..4:
  for exchange oldest -> newest:
    stop if it fits
    lower this exchange to that level

Every exchange reaches a level before any reaches the next, so a session that's slightly over loses detail from only its oldest exchanges, and a huge one keeps every user request it can before anything is omitted. The newest exchange is never omitted. Each exchange is rendered at each level once, so fitting is running arithmetic, not repeated estimates.

The previous summary still always stays in front. After a "too long" rejection the target now steps 85%, 70%, 50%, 35% (was 70%, 50%, 35%), and each retry refits from scratch, so a small overshoot costs a small cut.

Dropping tool definitions is safe here: the transcript has no real tool calls, only text lines, and a previous native compaction window only ever holds user messages and an encrypted compaction item. I checked that OpenAI (compact endpoint and trigger) accepts the window plus a transcript with no tools. The system prompt stays, since some routes may expect one.

Reasoning is no longer written into the kept recent messages. Real messages still replay reasoning as before.

Out of scope

  • Sending the kept recent messages as real messages instead of text.
  • Cutting a kept turn that's too big on its own between tool groups.
  • Summarizing the oldest history in several passes instead of omitting it.
  • A summary that stops on length.

Fixes #51818
Related: #43250


Added by the opencode bot (AI-generated).

Fixes #51818

Related: #43250

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transcript now shrinks in stages, keeping tool output whole, then cutting it to 1,250 and then 200 characters, and it only drops exchanges as a last resort. It works as described.

I ran the core session runner and compaction tests and the packages/core typecheck; all pass. I also ran the new runner tests against the base: the two new serialization cases fail there (an exchange is dropped where cutting tool output to 200 characters would have fit) and pass with this change. Dropping reasoning from SessionRecentText.messageToText also removes [Assistant reasoning] lines from the kept recent text, which is what #51818 reports.

The public API is unchanged. Only internal compaction request building changes; no SDK, HTTP, plugin, config or event surface is touched.

I have one question inline: whether every route accepts a transcript request with no system prompt.

Fixes #51818
Related: #43250

Comment thread packages/core/src/session/compaction.ts Outdated
omitted ? `[${omitted} older ${omitted === 1 ? "exchange" : "exchanges"} omitted]\n\n` : ""
const transcript = (exchanges: ReadonlyArray<string>, omitted: number) =>
LLMRequest.update(request, {
system: [],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With system: [], the Responses protocol omits instructions entirely. You checked the OpenAI compact endpoint and trigger, but this path also runs for ordinary summary streams on every provider. Have you confirmed that routes which have required a system prompt or instructions in the past (e.g. the ChatGPT sign-in route, or subscription/OAuth Anthropic) accept a request with none? If any of them reject it, the transcript fallback would fail exactly when it is needed. A minimal fixed instruction would avoid that risk.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit addresses my earlier question. The text transcript keeps the original system prompt and drops only the tool definitions, and the space calculation for dropping old exchanges now counts the system prompt. The core session runner and compaction tests (259) and the packages/core typecheck pass. The public API is unchanged. I have no further concerns.

@rekram1-node rekram1-node changed the title fix(core): shrink compaction transcripts in stages before dropping history fix(core): shorten compaction transcripts oldest first before dropping history Oct 22, 2026

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My earlier point about the system prompt stays addressed: the transcript still keeps it. The oldest-first shortening works as described. I checked it with the packages/core session runner and compaction tests (262 pass) and the typecheck (passes). The new and changed runner scenarios fail on the base and pass here. The public API is unchanged.

I have one small point inline: the note can undercount exchanges that disappear at the user-only level.

Comment thread packages/core/src/session/compaction.ts Outdated
}
if (total > room) return undefined

const omitted = levels.filter((level) => level === OMITTED).length

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An exchange whose user-only render is empty disappears from the transcript once it is lowered to USER_ONLY. That happens when its user message is only media, or when the first group starts with an assistant message. The note doesn't count it, because only level === OMITTED is counted here, so "[N older exchanges omitted]" undercounts. Counting exchanges whose chosen render is empty while renders[0] is not (or skipping straight to OMITTED for them) would keep the note accurate.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit addresses the note undercount: exchanges that become empty at the user-only level are now counted as omitted. The packages/core session runner and compaction tests (262) and the typecheck pass. The public API is unchanged. I have no further concerns.

@rekram1-node
rekram1-node merged commit ac68d6a into dev Oct 22, 2026
12 checks passed
@rekram1-node
rekram1-node deleted the staged-summary-shrink branch October 22, 2026 19:14
@opencode-agent opencode-agent Bot mentioned this pull request Oct 22, 2026
4 of 6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compaction keeps reasoning text in recent, which can make the context bigger than before compaction

1 participant