feat(prover-node): tear down checkpoint sub-tree once block proofs are ready (A-1213) - #24982
Open
PhilWindle wants to merge 2 commits into
Conversation
…ready Each CheckpointProver kept its sub-tree orchestrator — and every TxProvingState, AVM circuit input, and base/merge/parity proof it held — alive for the whole proof-submission window, long after the checkpoint's block proofs were produced. With a long submission window this accumulates across every proven checkpoint and dominates prover-node memory under load. Tear the sub-tree down as soon as its SubTreeResult is captured. The block-proof outputs survive via the resolved promise; post-completion consumers (the top-tree job, a rebuilt EpochSession, failure upload) read only whenBlockProofsReady() and the prover's own fields (checkpoint, txs, headers, sibling paths), never the sub-tree. Teardown is tracked so whenDone() awaits it and stays idempotent with the cancel/reap path.
whenDone() awaited runPromise (which resolves once block-level proving is merely enqueued) then an optional teardownPromise — but teardownPromise is set only when the sub-tree result lands, which happens after runPromise resolves. So a whenDone() call in that window returned while proving and teardown were still outstanding; the existing test masked it by awaiting whenBlockProofsReady() first. Await blockProofs settlement between runPromise and the teardown check: on success the result callback resolves blockProofs and synchronously sets teardownPromise before the await resumes, so teardown is observed; on failure/cancel blockProofs rejects and teardown is driven by the finally/cancel paths already awaited. Adds a regression test that calls whenDone() before the sub-tree result resolves and asserts it stays pending until stop() completes.
PhilWindle
force-pushed
the
phil/a-1213-release-intermediate-rollup-proofs-in-the-orchestrator-once
branch
from
July 28, 2026 11:07
b535c8e to
e80f746
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Reduces prover-node memory by tearing down each checkpoint's sub-tree orchestrator as soon as its block proofs are captured, instead of retaining it for the whole proof-submission window.
Each
CheckpointProverheld itsCheckpointSubTreeOrchestrator— and everyTxProvingState, AVM circuit input, and base/merge/parity proof it carried — alive until the prover was reaped at the end of the submission window, long after the checkpoint's block proofs were produced. Across every proven checkpoint this accumulates and dominates prover-node memory under load.Now, once the sub-tree's
SubTreeResultis captured, theCheckpointProverdrops the sub-tree orchestrator. The block-proof outputs survive via the resolved promise; post-completion consumers (the top-tree job, a rebuiltEpochSession, failure upload) read onlywhenBlockProofsReady()and the prover's own fields (checkpoint,txs, headers, sibling paths), never the sub-tree. Teardown is tracked sowhenDone()awaits it and stays idempotent with the cancel/reap path.Correctness
teardownSubTree()captures the orchestrator and nulls the field in a synchronous prefix before anyawait, so the success-driven teardown and a concurrent cancel/reap can't both callstop().whenDone()awaits the block-proof settlement betweenrunPromiseand the teardown check, so it can't report completion in the window where block-level proving has finished enqueueing (runPromiseresolved) but the sub-tree proofs — and the teardown they trigger — are still outstanding.Scope
This reclaims the cross-checkpoint retention of already-proven sub-trees. It does not free the intermediates of checkpoints that are still proving (the in-flight AVM inputs / base-rollup hints / recursive proofs). Freeing those as they are consumed is a separate, more invasive change and is deliberately left as a follow-up.
The broker remains a separate large consumer (A-1215), as does the prover-node's
BrokerCircuitProverFacadeinline-inputs retention (A-1517).Testing
yarn build, full@aztec/prover-nodecheckpoint-proversuite.whenDone()stays pending until the sub-tree result lands and teardown completes (asserted by calling it before the result resolves).