Skip to content

Delta: re-base onto the chain root instead of storing FULL at a chain's limits (#89) - #585

Merged
yifanxuaaa merged 2 commits into
layerfs_v2_0_0from
opt/89-chain-root-rebase
Nov 6, 2026
Merged

yifanxuaaa merged 2 commits into
layerfs_v2_0_0from
opt/89-chain-root-rebase

Conversation

@yifanxuaaa

Copy link
Copy Markdown
Contributor

When a delta chain hits its canonical budget or depth cap, the one PREFIX trial now runs against the chain's FULL root instead of storing the object FULL. The same rule applies to inode-branch programs at the metadata depth cap. This is a writer-only change: no format change, codec untouched. On fs-storage-quick, Q2 edit payload is −58.7%, its Commit-9 FULL spike (67,666 → 12,534 B) is gone, and Q6 new_metadata is −19.4%.

Fixes Ephemeral-AI-Lab/layerfs-project#89 (parent Ephemeral-AI-Lab/layerfs-project#84). Base: layerfs_v2_0_0 @ b4fa173e40.

Change

  • encoding/delta/select.rs:
    • After a candidate is chosen, trial_base reads its chain cost from the stored record sizes, the same sums acquisition charges.
    • If the chain is at the role's depth cap, or chain + target exceeds the 4×/2× cutoff budgets, the trial uses the chain root found by chain_root, a walk bounded by the maximum depth.
    • The replaced predecessor is never decoded. The root is acquired and authenticated, and the existing base-alone and post-acquire budget checks still apply to it.
    • A FULL candidate that doesn't fit still stores FULL.
  • encoding/delta/branch.rs: a predecessor at the metadata depth cap or over budget is replaced by the whole branch at its chain root, which must fit the same budgets.
  • Counters:
    • new rebased;
    • depth_capped now means "found at the cap" (every such candidate is re-based);
    • work_exceeded counts only refusals that stored FULL.
  • Docs: the handbook's chain-budget section, with the diagnostic figures.

Production LOC: 139048 -> 139127 (delta +79, all core).

Tests

  • layerfs-storage full suite passes.
  • Updated delta_depth_counts, chain_budget_scaling, delta_base_budget and inode_branch_deltas. They assert the re-base: depth 1, one trial, no FULL. Every version reads back through a fresh reader, which shows today's reader accepts every record written.
  • New tests:
    • a chunk chain at its cap re-bases;
    • one trial per dependent;
    • a corrupt root fails the Save;
    • 20 Commits of a branch chain store no whole branch.
  • inode_branch_delta_custody and sdk_large_file_budget pass. clippy and rustfmt are clean.

fs-storage-quick (campaign q89-c2, #89 only, against the pinned base baseline)

Row Metric Base (B) This PR (B) Δ
Q2 SQLite edit Commits 2–13 new_stored 290,078 119,925 −58.7%
Q2 SQLite Commit 9 new_stored 67,666 12,534 −81.5%
Q2 SQLite Σ payload (incl. create) 606,688 436,551 −28.0%
Q2 SQLite late file net 675,840 503,808 −25.5%
Q6 both Σ new_metadata 52,017 / 51,996 41,936 / 41,936 −19.4%
  • Inode-branch FULL spikes: gone (lane/10 8,527 → 2,757 B).
  • Root drift: one re-base against an aging root costs more (lane/18 1,111 → 2,982 B). It corrects itself once FULL wins again.
  • Other rows: within noise, including Q3/Q4, which are byte-identical. Every row's final state verified.

Read amplification (DIAGNOSTIC, in-process M03 save path): average final-version decode +22.5%, maximum 520,575 B, within the 524,288 B budget.

Contract notes:

  • A re-based predecessor is not authenticated during selection; its root is.
  • A depth-capped declared candidate re-bases instead of falling through to the ring.

🤖 Generated with Claude Code

yifanxuaaa and others added 2 commits November 6, 2026 13:10
…in's limits (#89)

When a payload candidate's chain is at the role's depth cap, or would exceed a
chain work budget with the new object on it (judged from the chain's stored
record sizes, the same sums acquisition charges), selection now runs its one
PREFIX trial against the FULL root of that chain instead of storing FULL. The
replaced candidate is never reconstructed; the root is acquired and
authenticated, and the root plus the target is held to the same budgets. The
same rule applies to inode-table branch programs at the metadata depth cap or
metadata budgets. Writer-only: the records are the existing PREFIX and branch
COPY/INSERT forms, the reader and codec are unchanged.

Counters: DeltaCounters.rebased and BranchCounters.rebased are new;
depth_capped now counts candidates found at the cap (all re-based), and
work_exceeded counts only budget refusals that stored FULL.

Tests: the depth-cap and work-budget tests now assert the re-base (depth 1,
one trial, rebased 1, no FULL) and read every version back through a fresh
reader; new tests cover a chunk chain at its cap, one trial per dependent and
a corrupt root failing the Save. Handbook sections 5.3, 5.4 and 6 updated.

Production LOC: 139048 -> 139127 (delta +79); core 38677 -> 38756 (+79).
Method: python3 tools/production_loc.py --root <git archive of parent HEAD>
vs --root <git archive of the staged tree>, combined scope, same counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the offline-model figures with the diagnostic measured over the
Storage save path (in-memory fixture, M03 sequence, one Save per Commit):
edit-Commit pack bytes 2,585,612 -> 1,201,562; final-version reconstruction
average 212,662 -> 260,555 canonical bytes, maximum 515,583 -> 520,575,
inside the 524,288-byte budget.

Production LOC: 139127 -> 139127 (delta 0); docs only.
Method: python3 tools/production_loc.py on parent and staged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yifanxuaaa
yifanxuaaa merged commit bdf10a2 into layerfs_v2_0_0 Nov 6, 2026
2 of 4 checks passed
@yifanxuaaa
yifanxuaaa deleted the opt/89-chain-root-rebase branch November 18, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant