Repository navigation
fix(search): recency boost goes to the matched chunk, gated by the all-words grade (#507) - #510
Merged
Merged
Conversation
…the all-words grade (#507) The recency leg ranked notes newest-first but credited each note's ord=0 title chunk whatever chunk had actually matched. With two same-titled notes the newer decoy's tied title match plus the boost beat the older note's full-body match. The boost now goes to a note's best-ranked content chunk (first in the keyword list, else the vector list). Candidates stay the notes whose title chunk a content leg returned; when any candidate also holds a chunk matching every query word, only those receive the boost. Epic #505 decision 4 (admit every matched note when the all-words grade is empty) and decision 7's fallback (no gate) each cost one bench case on the reference vault: a day-old note at keyword rank 88 took the rank-1 boost (recall@5 80.0 -> 76.7, MRR 0.6616 -> 0.6604). The narrower rule ranks all 30 cases identically to main. Hit.recency_rank is filled in and `omind search --explain` prints it. The strict xfail from 10.4.1 is removed; the decoy case passes under both indexing orders. Bumps to 10.4.2. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
Contributor
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 46 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #507
Part of #505 (last child)
What
A newer note that only shared the query's title could outrank an older note whose body matched every query word. The recency leg ranked notes newest-first but credited each note's
ord = 0title chunk whatever chunk had actually matched, so the real answer's keyword credit (rank 1, Details chunk) and its recency credit (title chunk) were split across two chunks, while the decoy's tied title match and its recency boost landed on one.require_allgrade, now returned by_keyword_leginstead of recomputed), only those candidates receive the boost. That is what drops the title-only decoy._W_RECENCYandRRF_Kare unchanged (decision 5).--explaincompletes the record (decision 6).Hit.recency_rankwas declared but never filled in; it is now, andomind search --explainprintsrecency=N.xfailfrom fix(search): deterministic index order and BM25 tie-breaks (#506) #509 is removed; the Kp12 decoy case passes under both forced indexing orders.Deviation from epic decision 4, with the measurement
Decision 4 said: when the all-words grade is empty, every matched note is eligible. I built that first and ran
omind bench --qualityon the reference vault (30 cases). It regressed: a note written the day before, matching at keyword rank 88, became the newest eligible note, took the rank-1 recency boost (0.25/61) and jumped from rank 6 to rank 2 for "how many pull requests before Codeberg throttles me", pushing the expected note from 5 to 6. Same shape on "what quality bar must the code I write meet" (21 → 22). Decision 7's fallback (best chunk, no gate at all) produced the identical regression.Root cause: the old eligibility rule, "the title chunk itself matched", was doing real work as a relevance gate. The all-words grade on a 1,789-note vault admits long chunks that merely contain every word. So this PR keeps the title-matched candidate set and narrows it with the all-words grade, rather than replacing it. That is a strict subset of the old eligibility plus the credit move.
omind bench --quality(reference vault, 30 cases)Per-case ranks for this PR are identical to main on all 30 cases.
Tests
test_recency_credits_the_matched_chunk_and_skips_a_title_only_decoy: the Kp12 pair with the decoy's filename sorting first (the fix(search): deterministic index order and BM25 tie-breaks (#506) #509 xfail arrangement). Asserts A first, A's hit is the Details chunk withrecency_rank == 1andkeyword_rank == 1, and the decoy'srecency_rank == 0.test_recency_boosts_every_name_match_when_no_chunk_matches_all_words: two notes named for one word each of a two-word query; both boosted, newest first, ranks 1 and 2.test_recency_ignores_a_note_the_query_names_nowhere_in_its_title: a body-only mention getsrecency_rank == 0, a note named for the query gets 1.test_search_explain_reports_the_recency_rank(CLI):--explainoutput containsrecency=1.test_recency_only_reorders_matches_it_never_adds_notesstill passes.a-to-zandz-to-a.Gates on this Mac (Python 3.11):
ruff check .clean,mypy srcstrict clean, full suite 4050 passed / 12 skipped / 1 xfailed (the pre-existing one).Bumps to 10.4.2; CHANGELOG, BACKLOG and docs/retrieval.md updated.
🤖 Generated with Claude Code
https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/