Drop the query planner statistics tables - #1027
Merged
Merged
Conversation
Contributor
|
| Project | Bencher |
| Branch | drop-planner-statistics |
| Testbed | intel-v1 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result microseconds (µs) (Result Δ%) | Upper Boundary microseconds (µs) (Limit %) |
|---|---|---|---|
| Adapter::Json | 📈 view plot 🚷 view threshold | 5.23 µs(+9.64%)Baseline: 4.77 µs | 5.65 µs (92.67%) |
| Adapter::Magic (JSON) | 📈 view plot 🚷 view threshold | 5.03 µs(+8.66%)Baseline: 4.63 µs | 5.40 µs (93.10%) |
| Adapter::Magic (Rust) | 📈 view plot 🚷 view threshold | 28.20 µs(+7.91%)Baseline: 26.13 µs | 28.56 µs (98.75%) |
| Adapter::Rust | 📈 view plot 🚷 view threshold | 4.68 µs(+25.70%)Baseline: 3.72 µs | 5.47 µs (85.52%) |
| Adapter::RustBench | 📈 view plot 🚷 view threshold | 4.68 µs(+25.82%)Baseline: 3.72 µs | 5.46 µs (85.71%) |
epompeii
marked this pull request as ready for review
September 14, 2026 04:10
epompeii
marked this pull request as draft
September 14, 2026 04:15
The 2026-08-17 migration ended with a limited ANALYZE. Its sampled statistics overstate how many rows share a key on indexes whose leading rows cluster on one value, and trusting them the planner scans the whole alert table for a status filtered alerts list and walks every head version for a perf query. A full ANALYZE keeps the alert scan and adds scans elsewhere. Without statistics the planner falls back on its structural heuristics, which pick the indexes. A new migration drops sqlite_stat1 and sqlite_stat4, with IF EXISTS because a database that never ran ANALYZE has neither and a build without ENABLE_STAT4 never creates the second. Its down is a comment: the statistics are not restored, since re-running a limited ANALYZE would reintroduce the problem. The 2026-08-17 migration is left as is. A bencher_schema test asserts that no sqlite_stat table exists after run_migrations. It failed before the new migration, since ANALYZE creates both tables even on an empty database, and passes with it.
epompeii
force-pushed
the
drop-planner-statistics
branch
from
September 14, 2026 04:20
9b8072b to
4cacc3e
Compare
epompeii
marked this pull request as ready for review
September 14, 2026 04:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
2026-09-14-120000_drop_planner_statisticsdropssqlite_stat1andsqlite_stat4(DROP TABLE IF EXISTS, since a database that never ranANALYZEhas neither and a build withoutENABLE_STAT4never creates the second). Itsdown.sqlis a comment: the statistics are not restored, because re-running a limitedANALYZEwould reintroduce the problem. The 2026-08-17 migration is left as is; applied migrations are history.migrations_leave_no_planner_statistics_behindinbencher_schemaasserts that nosqlite_stat%table exists afterrun_migrations.AGENTS.md, next to the other database rules: migrations never runANALYZEorPRAGMA optimize, and nothing creates thesqlite_stattables; the query planner runs without statistics.Pending v0.6.13section.Why
The 2026-08-17 migration ended with a limited
ANALYZE. The sampled statistics overstate how many rows share a key on indexes whose leading rows cluster on one value. Trusting them, the planner scans the whole alert table for a status filtered alerts list and walks every head version for a perf query. A fullANALYZEdoes not help: it keeps the alert scan and adds scans on other query shapes. Without statistics the planner falls back on its structural heuristics, which pick the indexes, as it did before the statistics were introduced.How it was tested
develbefore the migration existed and failed withfound ["sqlite_stat1", "sqlite_stat4"], sinceANALYZEcreates both tables even on an empty database. With the migration it passes.lib/api_projects/tests/metric_migration.rsreverts and reapplies the migrations, so the comment-onlydown.sqland the re-drop after the re-runANALYZEare both exercised.cargo nextest run --all-features --profile ci,cargo test --doc --all-features,cargo clippy --no-deps --all-targets --all-features -- -Dwarnings,cargo check --no-default-features,cargo fmt --all -- --check.