fix(server): paginate large thread history to stop the server running out of memory

[olafura]

Fixes the server-side root cause behind #2761 (and likely #996).

What Changed

The server used to load a thread's entire history — every tool activity, message, and checkpoint — into memory at once. On a busy database that's hundreds of MB, which exhausted the Node heap and crashed the server (and made large threads slow to load / churn reconnects, as reported in #2761).

Now the server loads only the most recent slice of a thread's activity and tells the client whether older history exists; the web and mobile apps fetch older pages on demand as you scroll up. Deep history is still fully reachable — it's paged in instead of loaded all at once.

• Server: the snapshot endpoint and the t3 CLI offline path stop loading every thread's full history (they only read the project list). Thread detail is bounded to the most recent activities, with a hasMoreActivities flag and a cursor-paginated getThreadActivities RPC for older pages. The older-page query is shaped to use the existing index instead of a full sort.
• Web / Mobile: lazy-load older history on scroll-up — prepend older pages, anchored so the viewport doesn't jump.

Three commits, one per layer (server / web / mobile), so the mobile commit can be dropped if you'd prefer a smaller PR.

Why

Programs in a thread emit a lot of tool activity over time. The server treated "open a thread" (and "load the snapshot") as "read all of it from SQLite into memory." For a long-lived thread or a busy database, that single read materialised more than the heap could hold, so the process ran out of memory and died — and even when it didn't crash, shipping one giant snapshot blocked the connection (#2761). Bounding the read + paging older history on demand keeps memory flat regardless of how much history a thread accumulates.

UI Changes

The web and mobile timelines gain a small "Load older history" affordance and a loading spinner when you scroll to the top of a thread longer than the window. It's a minor, additive interaction. I wasn't able to attach before/after screenshots or a scroll video here — happy to add them, or the server commit alone (no UI) carries the actual reliability fix if you'd rather review that first.

Checklist

• I explained what changed and why
• before/after screenshots for UI changes — to be added (web/mobile lazy-load affordance)
• video for the scroll-up interaction — to be added
• The server fix is independently reviewable (first commit, no UI)

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes core read paths and pagination semantics for long threads (including legacy unsequenced rows); memory/OOM fix is high impact but clients must page correctly to see full history.

Overview
Server no longer materializes full thread activity history on open. Thread detail returns at most 500 recent activities plus hasMoreActivities, with a new getThreadActivities cursor RPC (beforeSequence or legacy unsequenced beforeCreatedAt/beforeActivityId). Orchestration snapshot and t3 CLI offline project resolution use getCommandReadModel instead of the full snapshot to avoid loading huge per-thread tables.

Web and mobile merge lazily fetched older pages into the feed (dedup, in-flight guards, reset on window reshape) using shared helpers liveWindowOldestActivityId / oldestActivityByChronology. Timelines load more at scroll-top (maintainVisibleContentPosition, header spinner or “Load older history” on web).

Behavior: threads with >500 activities need backward paging for older tool history; initial detail is a window, not the full log.

^{Reviewed by Cursor Bugbot for commit 311b469. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Paginate large thread history to prevent server memory exhaustion

• Thread detail queries now load at most 500 recent activities (plus one to detect overflow), with hasMoreActivities set on OrchestrationThread when older entries exist.
• Adds a new getThreadActivitiesPage RPC (orchestration.getThreadActivities) that pages older activities using either a beforeSequence or beforeCreatedAt/beforeActivityId cursor.
• Web (MessagesTimeline) and mobile (ThreadFeed) UIs display a 'Load older history' button or spinner at the top of the feed and trigger paging when scrolled to the start.
• Client state (ChatViewContent and useThreadComposerState) manages pagination state, deduplicates pages, and resets older-page history when the live activity window reshapes.
• Behavioral Change: threads with more than 500 activities will no longer return the full history in the initial snapshot; clients must page backwards to retrieve older entries.

^{Macroscope summarized 311b469.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 2720bf69-78b8-4244-aef1-6681736d7d3b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[olafura]

Thanks Bugbot — addressed in cddc7e1:

• Reconnect leaves stale older pages / Revert keeps lazy-loaded activities (both): the lazy-load state now resets when the live window's oldest activity id changes, not just on thread switch. That id is stable while activities only append, so it doesn't fire during a normal turn — but a reconnect re-snapshot or a checkpoint revert that drops live rows changes it, invalidating the prepended older pages. Applied to web and mobile.
• Web skips live activity dedup: web now dedups a new older page against both the already-loaded older pages and the live window (mobile already did this), so a boundary overlap can't produce duplicate ids.
• Unsequenced cursor hides sequenced history: I believe this is a false positive. Legacy NULL-sequence rows always sort as the oldest in both the detail window (sequence DESC, NULLs last) and the pager. So the oldest-loaded row becomes unsequenced only after every sequenced row is already in the loaded set — there are never sequenced rows older than an unsequenced one for the sequence IS NULL cursor to strand. Happy to add a test if useful.

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a new pagination system for thread activities across server, web, and mobile with significant runtime behavior changes. An unresolved bug report identifies potential stale state after checkpoint reverts, requiring human attention.

^{You can customize Macroscope's approvability policy. Learn more.}

[olafura]

Addressed the latest review in 75d25e266 (rebased on current main):

• Stale in-flight load after reset — the reset now bumps a generation counter (olderActivitiesGenRef); the in-flight load captures it at dispatch and drops its result in .then()/.finally() if it changed, so a late page can't repopulate the cleared state. Replaces the thread-key-only staleness check (which was unchanged on a same-thread reshape).
• Revert may skip reset / Reconnect leaves stale pages — the reset now also fires when the live window shrinks (a checkpoint revert that drops rows without changing the oldest row), not just when the oldest id changes (reconnect). A pure append (same oldest, larger count) keeps the loaded pages. Applied to web and mobile.
• Empty page churns / loops — when dedup leaves nothing new, the handler returns without creating a new array reference and stops, so an unadvanced cursor can't loop identical requests.
• Web skips live dedup — dedups against the full merged set (older + live).
• Unsequenced cursor hides sequenced history — I believe this is a non-issue and added a server regression test to prove it: NULL-sequence rows always sort oldest, so when the oldest loaded row is unsequenced every sequenced row is already in the window; the sequence IS NULL cursor reaches all 103 older unsequenced rows in the test without stranding any of the sequenced ones.

Server/web/mobile typecheck, lint, and the projection + timeline suites pass.

[olafura]

Addressed the latest review round:

Cursor — "Append clears lazy-loaded history" (Medium) — The reset effect used liveThreadActivities[0]?.id as the live window's oldest boundary. The reducer's activityOrder sorts unsequenced rows to the end (a missing sequence is treated as newest) while the server snapshot lists legacy unsequenced rows first, so the first live append re-sorted the array and shifted index 0 — making a plain append look like a window reshape and wrongly clearing the user's scrolled-up history. Replaced it with an order-independent oldest sentinel, liveWindowOldestActivityId (min by createdAt, then id), which is stable across appends (always newest) and only changes on a genuine reshape (reconnect re-snapshot / revert). Applied to both ChatView and the mobile composer; the helper lives in client-runtime with unit tests.

macroscope — mixed-thread flash (Low, ChatView.tsx:1724) — Moved the older-pages reset from useEffect to useLayoutEffect so the cleared state commits before paint; otherwise a thread switch rendered one frame with the previous thread's lazy-loaded pages still merged in, flashing stale work-log/approval rows. Mirrored in mobile.

The two earlier macroscope Mediums (1703 staleness guard, 1762/1786 empty-fresh loop) are already ✅ resolved in the prior commit — the generation-ref guard supersedes the request-key check, and the empty-fresh path early-returns without churning the array. The High 1772 ("unsequenced cursor hides sequenced history") remains a false positive: the lazy-load cursor (threadActivities[0]) is only unsequenced when the whole thread fits the window, in which case hasMoreActivities is false and no sequenced rows are stranded.

pnpm --filter @t3tools/web typecheck, --filter @t3tools/mobile typecheck, the client-runtime reducer suite (24 passing incl. the new helper specs), and lint are green. Squashed into the lazy-load-hardening commit and rebased on upstream/main.

[cursor]

Stale hasMore after revert

Medium Severity

After a checkpoint revert, the thread reducer drops activities but keeps the prior hasMoreActivities value. Lazy-load reset clears prepended pages yet hasMoreOlderActivities still reads that stale flag when no older page was loaded, so the UI can offer older history or call pagination when the retained set no longer has anything beyond the window.

Additional Locations (2)

• apps/web/src/components/ChatView.tsx#L1775-L1778
• apps/mobile/src/state/use-thread-composer-state.ts#L171-L174

 

^{Reviewed by Cursor Bugbot for commit 618df70. Configure here.}

[olafura]

Took another careful look — this is a false positive. thread.reverted keeps activities matching turnId === null || retainedTurnIds.has(turnId), i.e. it drops activities for the reverted (newer) turns and keeps the older/retained ones. hasMoreActivities describes whether rows exist older than the window's oldest boundary, which a revert of newer turns does not change — so the older rows it advertises still exist on the server and "load older" remains correct. And even in the worst case where the flag were momentarily stale-true, the lazy-load no-ops safely: an empty/all-overlap page now respects the server's page.hasMore (see the cursor fix in ead248b4) and stops without looping. No code change needed here.

[olafura]

Latest round:

Cursor — "Wrong pagination cursor after live sort" (High). Valid consistency gap I introduced last round. I'd made the reshape sentinel order-independent (liveWindowOldestActivityId = chronological-oldest) but left the pagination cursor on threadActivities[0], which the reducer can fill with a newer sequenced row (it sorts unsequenced rows to the end). Those two now agree: extracted oldestActivityByChronology(activities) and made the cursor page from it in both ChatView and the mobile composer, so paging always continues from the genuinely-oldest loaded row (driving the unsequenced beforeCreatedAt/beforeActivityId cursor when that row is legacy-unsequenced, instead of a beforeSequence that could skip it). The sentinel now delegates to the same helper. Added a reducer test asserting the cursor-oldest is the unsequenced legacy row and that the sentinel agrees.

Cursor — "Stale hasMore after revert" (Medium). False positive. thread.reverted keeps activity.turnId === null || retainedTurnIds.has(activity.turnId) — it drops activities for the reverted (newer) turns, never the older history. hasMoreActivities describes whether older rows exist beyond the window, which a revert of newer turns doesn't change, so retaining the flag is correct. (And even if it were stale-true, the lazy-load already no-ops on an empty page: fresh.length === 0 clears hasMore without looping.)

@t3tools/web + @t3tools/mobile + @t3tools/client-runtime typecheck clean, reducer suite 26 passing, lint clean. Squashed into the lazy-load-hardening commit and rebased on upstream/main.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c8e6474. Configure here.}

[olafura]

Self-review (10-angle, max effort) of 848cff64

Ran an adversarial multi-angle review and grounded the findings against the real 1.5 GB database (131,532 activities). Posting the results.

Key empirical finding

The projection_thread_activities.sequence column is 100% NULL in every database I can observe (both the 4 MB dev DB and the 1.5 GB realistic one), and 0 threads have a NULL-sequence row newer than a sequenced one. runtimeEventToActivities only sets sequence from event.sessionSequence, which no provider currently emits. So the beforeSequence/sequenced-row pathway is never exercised in production — all paging goes through the created_at keyset. This reframes the scary-looking findings as latent, not live.

Fixed

Sev	Finding	Fix
Low (hardening)	The internal SQL-input schemas typed beforeSequence/limit as bare Schema.Number, looser than the contract's NonNegativeInt. The WHERE clause (sequence < beforeSequence OR sequence IS NULL) is only equivalent to the old COALESCE(sequence,-1) < beforeSequence for non-negative cursors — a negative one silently returns only unsequenced rows.	Tightened both internal schemas to NonNegativeInt so any future non-RPC caller is validated, not just the RPC boundary. Typecheck + 11 pager tests green.

Latent / by-design (NOT changed — flagging for a decision)

• Window vs reducer NULL-ordering disagreement (highest-severity-if-triggered): the server window orders sequence DESC (SQLite → NULLs last/oldest) while the client reducer's activityOrder uses sequence ?? MAX_SAFE_INTEGER (NULLs newest). They only agree today because every row is NULL. If sequence is ever partially backfilled, a thread with mixed rows would (a) drop the newest unsequenced rows from the 500-window and (b) paginate in a different order than it displays. The robust fix is to order both sides by created_at consistently — but that's an index-strategy change (the covering index leads with sequence) and a no-op today, so I've left it. Recommend either ordering by created_at end-to-end, or guaranteeing sequence is all-or-nothing.
• Speculative sequence machinery: given the above, the entire sequenced-cursor arm (second SQL query, union input arm, "beforeSequence" in input branch, the sequence-leading index) is dead in practice. A single (created_at, activity_id) keyset would cover 100% of today's reality with ~half the surface. Kept as forward-looking, but worth a conscious call.
• web/mobile duplication (AGENTS.md "Duplicate logic across multiple files … should be avoided"): the ~120-line lazy-load state machine (gen guard, in-flight ref, reshape useLayoutEffect, dedup) is near-verbatim in ChatView.tsx and use-thread-composer-state.ts. Should become one shared hook — deferred because it's a cross-app React-architecture refactor, not a surgical fix.
• Asymmetry: activities are windowed to 500 but messages in the same getThreadDetailById are still unbounded, so the heap-OOM rationale is only half-applied.
• handleScroll/loadOlder identity churn on every live append (perf, not correctness); THREAD_DETAIL_ACTIVITY_WINDOW overloaded as window size + default page + max page.

Verified safe (no action)

getSnapshot→getCommandReadModel (only the CLI reads .projects); toReversed() (Node/lib supports it); mapThreadActivityRow typing; hasMoreActivities optional doesn't break fixtures/decoders; window slice/hasMore arithmetic; the empty-page stop guard; sequence===0 cursor; created_at tie-break cursor. No CLAUDE.md/AGENTS.md violations beyond the duplication note.

[olafura]

Addressed the latest review round (ead248b4)

Three threads — two fixed, one verified as a false positive. Both fixes applied to web (ChatView) and mobile (use-thread-composer-state) identically.

🟡 Cursor — "Duplicate page ends pagination early" (r3455736449) — Fixed. Paging now runs off an explicit olderCursorRef that advances to each page's oldest row (pages are ascending, so [0] is oldest) on every successful page — even one that dedupes to zero new rows. Because the server cursor is strict, the cursor strictly decreases, so an all-overlap page keeps paging (respecting the server's page.hasMore) without re-requesting the same cursor or looping. The old setOlderHasMore(false) early-stop is removed.

🟢 macroscope — "stale dedup seen" (r3455790594) — Fixed as suggested. Added a threadActivitiesRef (mobile: mergedActivitiesRef) assigned the merged array every render; the .then() dedup now builds seen from …Ref.current instead of the closure-captured snapshot, so a live append or a prior prepend that settles mid-flight is always reflected.

🟡 Cursor — "Stale hasMore after revert" (r3455516138) — False positive. thread.reverted keeps turnId === null || retainedTurnIds.has(turnId), dropping activities for the reverted (newer) turns and keeping older ones. hasMoreActivities describes whether rows exist older than the window, which a newer-turn revert doesn't change — so retaining it is correct. No code change.

These were latent (reachable only in the mixed sequenced+unsequenced case, which never occurs — sequence is 100% NULL in the real DB), but the fixes make paging correct-by-construction rather than relying on that invariant.

Verified: @t3tools/web + @t3tools/mobile typecheck clean, lint clean, reducer (26) / MessagesTimeline (12) / pager (11) suites green. Squashed into the lazy-load commit and rebased on upstream/main.