Repository navigation
PostgreSQL Init: empty the acquisition tables when the last operation is released (#87) - #582
Merged
Merged
Conversation
… is released (#87) Deleted Init working rows stayed in init_entry/init_host_file as dead tuples until autovacuum, and their B-tree pages were never returned, so a PostgreSQL Store was several MB larger right after Init than a minute later and kept the emptied indexes for good. The transaction that releases the last operation (release, or the dispose job that removes the last row) now learns from the RELEASE statement whether any other record remains and, if none does, empties the three acquisition tables with one TRUNCATE ... CONTINUE IDENTITY inside a savepoint. It runs under the lane row every acquisition write holds, so no operation can begin until it commits; locks are taken record first (the order units read them) with a 2 s lock_timeout, past the default deadlock_timeout that cancels an autovacuum holding a table. A definite refusal rolls back to the savepoint and the release commits unchanged; a lost reply stays uncertain. Nothing is retried. The file_root UPDATE (writes.rs FILE_ROOTS) is unchanged: removing it needs an acquisition schema change (reported to the lead). Tests: postgresql_acquisition_compaction (Store bytes right after Init equal a never-used Store's and are unchanged by VACUUM; another session's live operation keeps its rows; a held table keeps the release; three concurrent sessions x 12 Inits). Production LOC: 139048 -> 139062 (delta +14). Scope/method: combined production scope of `python3 tools/production_loc.py --root <tree>` on `git archive` of the parent (HEAD) and of the staged tree (`git write-tree`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mengmengmengqiang
pushed a commit
to tokenaissance/layerfs
that referenced
this pull request
Nov 9, 2026
Brings in Ephemeral-AI-Lab#582-Ephemeral-AI-Lab#593 (SQLite idle WAL checkpoint, ring admission, chain-root rebase, PG Init cleanup). One conflict: api/crates/layerfs-control/src/store/open.rs, both sides only added functions (#64 holds; Ephemeral-AI-Lab#86 idle-checkpoint helpers); kept both. Production LOC: 139821 -> 140178 (delta +357), first-parent comparison (pg-r2 76a32a9 -> merged tree), counted with tools/production_loc.py; the delta is the upstream changes. Checked on the merged tree: Clippy -D warnings (persistence, store-service, daemon, control, storage, content, workspace, history, bridge with postgresql/machine) and rustfmt clean; 219 test binaries of storage, content, workspace, persistence and store-service run one by one against PostgreSQL 17: 218 pass (1,020 tests), filesystem_attributes fails 1/13 only because its fixture path is relative to the crate directory (known, same at base). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PostgreSQL Store size after
project initno longer depends on autovacuum. The transaction that releases a session's last Init operation now also empties the Init working tables. On fs-storage-quick Q6 PostgreSQL, the Store is the same size immediately and after autovacuum: 28,917,760 B, against 38,158,336 B / 31,358,976 B on base.Fixes Ephemeral-AI-Lab/layerfs-project#87 (parent Ephemeral-AI-Lab/layerfs-project#84). Base:
layerfs_v2_0_0@b4fa173e40.Change
engine/crates/layerfs-persistence/src/backend/postgresql/acquisition/cleanup.rs:SAVEPOINT,SET LOCAL lock_timeout 2000,LOCK … ACCESS EXCLUSIVE,TRUNCATE init_operation, init_entry, init_host_file CONTINUE IDENTITY. The record table is locked first, in the same order units take the tables.Why it is safe:
begin, holds thestore_lanerow, so no operation can start until the truncate commits.CONTINUE IDENTITYmeans operation ids and epochs are never reused.SET LOCALtimeout reaches nothing else.Not in this PR: the issue's second item, writing
file_rootat insert. It needs a change toacquisition.sql, the recorded schema, so it is a format change; it moves to #90's schema-v2 work. With the truncate, the UPDATE's dead tuples last only while an Init runs.Production LOC: 139048 -> 139062 (delta +14).
Tests
tests/postgresql_acquisition_compaction.rs: 4/4 pass, and all 4 fail on the basecleanup.rs. It covers:postgresql_open_check,postgresql_identical_ids,postgresql_commit_round_trips,diagnostics_during_lane;init_postgresql;postgresql_openandpostgresql_open_cost.-D warningsand rustfmt are clean.fs-storage-quick (campaign q87-c1, Docker, both backends, late probe 75 s, against the pinned base baseline)
init_*tables are 24,576 / 24,576 / 16,384 B with 0 dead tuples, the same as a fresh Store. Base still held 950,272 + 1,474,560 B there.No format or codec change.
🤖 Generated with Claude Code