Skip to content

perf: skip provably ambiguous XPath selector searches - #62

Merged
egorsmkv merged 1 commit into
masterfrom
optimize-xpath
Oct 7, 2026
Merged

egorsmkv merged 1 commit into
masterfrom
optimize-xpath

Conversation

@egorsmkv

@egorsmkv egorsmkv commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Mirrored, attribute-heavy branches currently exhaust all 256 XPath candidates even though none can distinguish the target. A bounded preflight on the normalized XPath tree now proves ambiguity when another node shares the target's tag, full attribute map and sibling position, and contains its ancestor shapes as an ordered subsequence. When that proof succeeds, generation immediately returns the already validated absolute fallback; otherwise the existing Robula-style search, ranking and budgets run unchanged.

The preflight applies only to searches larger than the 128-entry compilation cache and caps shape comparisons at the candidate count. There are no dependency or API changes.

Coverage

  • Regression fixtures for attrs-128 and deep-attrs, including repeated generation and target identity.
  • Attribute-heavy ancestor anchors remain robust to inserted wrappers and reordered siblings; differing sibling positions retain positional selectors.
  • A deterministic Rust regression checks that both heavy fixtures compile/evaluate zero candidates, checks preflight budget exhaustion, and independently evaluates every generated candidate to guard the ambiguity proof against future grammar changes.
  • Benchmark repeats now verify selector stability and record the returned selector.

Measurements

Full selector benchmark: all seven scenarios, both CSS and XPath, 15 warm repeats per scenario, release build, CPython 3.12.14, AMD EPYC 9V74. These local before/after measurements use baseline 2633b5a; the patch was subsequently rebased onto current v0.14.0 master (8927546). The fixture data-attribute candidate grammar is unchanged by the intervening href/src ranking update.

XPath scenario Before median, ms After median, ms
deep-16 0.309 0.326
deep-64 1.337 1.560
deep-128 2.868 2.777
attrs-8 50.532 0.101
attrs-32 65.299 0.131
attrs-128 155.157 0.360
deep-attrs 304.223 0.322

Profiling attributes most latency to the 227 ancestor candidates: attrs-128 spends approximately 131 ms executing them and 25 ms compiling them; deep-attrs spends approximately 278 ms executing them and 26 ms compiling them. The shortcut removes these evaluations for provably indistinguishable branches. Distinguishable documents retain the original search plus bounded preflight overhead; these are not general speedup guarantees for arbitrary HTML.

Validation on current master

  • Release extension build: passed.
  • Python suite: 69 passed.
  • Rust library suite: 31 passed.
  • Rust formatting, Ruff checks/formatting for changed Python files, and diff whitespace checks: passed.
  • Clippy with -D warnings -A clippy::collapsible_if: passed. Unmodified strict Clippy reports two pre-existing collapsible_if lints in the href/src ranking code added by Prefer scraping-stable href/src attributes in XPath generation #61, outside this patch.

The preflight proof must be updated if the candidate grammar later adds text predicates, ancestor positions, or other distinguishing transformations.

@egorsmkv
egorsmkv merged commit ea50c37 into master Oct 7, 2026
9 checks passed
@egorsmkv
egorsmkv deleted the optimize-xpath branch October 7, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant