Branch Name
main
Commit ID
343b1a7 (PR #25527 merge result; the failing path is unchanged from main)
Environment
Actual Behavior
data branch diff t2 against t0 output summary intermittently fails with:
internal error: cannot find database 282762 (test_chain_diff) created_time at snapshot 1783954047824351071-1
The database was committed at 1783954047186529290-0, well before the query snapshot. In the same CN, transaction, and snapshot, the first two mo_database snapshot scans each read 5 rows and found database 282762. A third scan read only 1 row and reported the database missing.
Expected Behavior
Snapshot scans must return a consistent view while an appendable object is being flushed to a persisted object. The data branch query must not intermittently lose committed catalog rows.
Steps to Reproduce
Run test/distributed/cases/git4data/branch/diff/diff_12.sql in standalone multi-CN + proxy BVT while catalog flush is active. The failure is timing dependent.
Root Cause
scanSnapshotRelationByID obtains current ranges from Relation.Ranges(). The returned relData embeds the immutable PartitionState used to build those ranges. However, ScanSnapshotWithCurrentRanges discards that state and calls tbl.getPartitionState() again for in-memory rows; CollectTombstones() also resolves the latest state independently.
A flush uses copy-on-write to swap the partition state. This permits a mixed view: ranges from the pre-flush state, but rows and tombstones from the post-flush state. The failing timeline proves this exact window:
- 14:47:27.821903:
[FT-11499]1-mo_database flush starts.
- 14:47:27.829672: ranges contain only the in-memory marker;
disk-block-cnt=0.
- 14:47:27.829905 and 14:47:27.830842: scans on the pre-flush state read 5 rows and find the database.
- 14:47:27.830680: third scan starts with the same pre-flush ranges.
- 14:47:27.830758:
mo_database flush ends and swaps state.
- 14:47:27.834885: third scan uses the newer state, reads only 1 row, and misses database 282762.
The fix should use the PartitionState embedded in relData consistently for persisted ranges, committed in-memory rows, and committed tombstones.
Additional Information
This is unrelated to PR #25527. That PR only changes prepared-expression JSON comparison paths in frontend/planner/colexec and does not touch data branch, catalog, logtail, snapshot scan, or storage code.
Branch Name
main
Commit ID
343b1a7 (PR #25527 merge result; the failing path is unchanged from main)
Environment
test/distributed/cases/git4data/branch/diff/diff_12.sql, line 249Actual Behavior
data branch diff t2 against t0 output summaryintermittently fails with:The database was committed at
1783954047186529290-0, well before the query snapshot. In the same CN, transaction, and snapshot, the first twomo_databasesnapshot scans each read 5 rows and found database 282762. A third scan read only 1 row and reported the database missing.Expected Behavior
Snapshot scans must return a consistent view while an appendable object is being flushed to a persisted object. The data branch query must not intermittently lose committed catalog rows.
Steps to Reproduce
Run
test/distributed/cases/git4data/branch/diff/diff_12.sqlin standalone multi-CN + proxy BVT while catalog flush is active. The failure is timing dependent.Root Cause
scanSnapshotRelationByIDobtains current ranges fromRelation.Ranges(). The returnedrelDataembeds the immutablePartitionStateused to build those ranges. However,ScanSnapshotWithCurrentRangesdiscards that state and callstbl.getPartitionState()again for in-memory rows;CollectTombstones()also resolves the latest state independently.A flush uses copy-on-write to swap the partition state. This permits a mixed view: ranges from the pre-flush state, but rows and tombstones from the post-flush state. The failing timeline proves this exact window:
[FT-11499]1-mo_databaseflush starts.disk-block-cnt=0.mo_databaseflush ends and swaps state.The fix should use the
PartitionStateembedded inrelDataconsistently for persisted ranges, committed in-memory rows, and committed tombstones.Additional Information
This is unrelated to PR #25527. That PR only changes prepared-expression JSON comparison paths in frontend/planner/colexec and does not touch data branch, catalog, logtail, snapshot scan, or storage code.