Skip to content

fix(search): recency boost goes to the matched chunk, gated by the all-words grade (#507) - #510

Merged
CryptoJones merged 1 commit into
mainfrom
fix/505-recency-best-chunk
Oct 31, 2026
Merged

CryptoJones merged 1 commit into
mainfrom
fix/505-recency-best-chunk

Conversation

@CryptoJones

Copy link
Copy Markdown
Owner

Closes #507
Part of #505 (last child)

What

A newer note that only shared the query's title could outrank an older note whose body matched every query word. The recency leg ranked notes newest-first but credited each note's ord = 0 title chunk whatever chunk had actually matched, so the real answer's keyword credit (rank 1, Details chunk) and its recency credit (title chunk) were split across two chunks, while the decoy's tied title match and its recency boost landed on one.

  • Credit moves to the matched chunk. The boost goes to a note's best-ranked content chunk: its first chunk in the keyword list, otherwise in the vector list (epic decision 4, option 3 part).
  • Gate. Candidates stay what they were: notes whose title chunk a content leg returned. When any candidate also holds a chunk matching every query word (the require_all grade, now returned by _keyword_leg instead of recomputed), only those candidates receive the boost. That is what drops the title-only decoy.
  • Recency still only reorders. A note no content leg returned has no best chunk and cannot be added. _W_RECENCY and RRF_K are unchanged (decision 5).
  • --explain completes the record (decision 6). Hit.recency_rank was declared but never filled in; it is now, and omind search --explain prints recency=N.
  • The strict xfail from fix(search): deterministic index order and BM25 tie-breaks (#506) #509 is removed; the Kp12 decoy case passes under both forced indexing orders.

Deviation from epic decision 4, with the measurement

Decision 4 said: when the all-words grade is empty, every matched note is eligible. I built that first and ran omind bench --quality on the reference vault (30 cases). It regressed: a note written the day before, matching at keyword rank 88, became the newest eligible note, took the rank-1 recency boost (0.25/61) and jumped from rank 6 to rank 2 for "how many pull requests before Codeberg throttles me", pushing the expected note from 5 to 6. Same shape on "what quality bar must the code I write meet" (21 → 22). Decision 7's fallback (best chunk, no gate at all) produced the identical regression.

Root cause: the old eligibility rule, "the title chunk itself matched", was doing real work as a relevance gate. The all-words grade on a 1,789-note vault admits long chunks that merely contain every word. So this PR keeps the title-matched candidate set and narrows it with the all-words grade, rather than replacing it. That is a strict subset of the old eligibility plus the credit move.

omind bench --quality (reference vault, 30 cases) recall@1 recall@5 MRR
main (29e96a2) and #509 (f8cc7f1) 56.67 80.00 0.6616
decision 4 as written 56.67 76.67 0.6604
decision 7 fallback (no gate) 56.67 76.67 0.6604
this PR 56.67 80.00 0.6616

Per-case ranks for this PR are identical to main on all 30 cases.

Tests

  • test_recency_credits_the_matched_chunk_and_skips_a_title_only_decoy: the Kp12 pair with the decoy's filename sorting first (the fix(search): deterministic index order and BM25 tie-breaks (#506) #509 xfail arrangement). Asserts A first, A's hit is the Details chunk with recency_rank == 1 and keyword_rank == 1, and the decoy's recency_rank == 0.
  • test_recency_boosts_every_name_match_when_no_chunk_matches_all_words: two notes named for one word each of a two-word query; both boosted, newest first, ranks 1 and 2.
  • test_recency_ignores_a_note_the_query_names_nowhere_in_its_title: a body-only mention gets recency_rank == 0, a note named for the query gets 1.
  • test_search_explain_reports_the_recency_rank (CLI): --explain output contains recency=1.
  • test_recency_only_reorders_matches_it_never_adds_notes still passes.
  • The fix(search): deterministic index order and BM25 tie-breaks (#506) #509 decoy case is a plain passing test under a-to-z and z-to-a.

Gates on this Mac (Python 3.11): ruff check . clean, mypy src strict clean, full suite 4050 passed / 12 skipped / 1 xfailed (the pre-existing one).

Bumps to 10.4.2; CHANGELOG, BACKLOG and docs/retrieval.md updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

…the all-words grade (#507)

The recency leg ranked notes newest-first but credited each note's ord=0
title chunk whatever chunk had actually matched. With two same-titled
notes the newer decoy's tied title match plus the boost beat the older
note's full-body match.

The boost now goes to a note's best-ranked content chunk (first in the
keyword list, else the vector list). Candidates stay the notes whose title
chunk a content leg returned; when any candidate also holds a chunk
matching every query word, only those receive the boost.

Epic #505 decision 4 (admit every matched note when the all-words grade
is empty) and decision 7's fallback (no gate) each cost one bench case on
the reference vault: a day-old note at keyword rank 88 took the rank-1
boost (recall@5 80.0 -> 76.7, MRR 0.6616 -> 0.6604). The narrower rule
ranks all 30 cases identically to main.

Hit.recency_rank is filled in and `omind search --explain` prints it.
The strict xfail from 10.4.1 is removed; the decoy case passes under both
indexing orders.

Bumps to 10.4.2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
@coderabbitai

coderabbitai Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 46 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 883ec1d3-2727-4f01-a443-002054d91fdb
Commits

Reviewing files that changed from the base of the PR and between f8cc7f1 and a0459b9.

Files ignored due to path filters: 1
  • uv.lock is excluded by !**/*.lock
Files selected for processing: 10
  • BACKLOG.md
  • CHANGELOG.md
  • docs/retrieval.md
  • pyproject.toml
  • src/omind/__init__.py
  • src/omind/cli.py
  • src/omind/searchindex.py
  • tests/test_cli.py
  • tests/test_name_timelines.py
  • tests/test_searchindex.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@CryptoJones
CryptoJones merged commit 60e093d into main Oct 31, 2026
22 checks passed
@CryptoJones
CryptoJones deleted the fix/505-recency-best-chunk branch October 31, 2026 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recency must not lift a newer note's title-only match above a note whose body matches every query word

1 participant