diff --git a/BACKLOG.md b/BACKLOG.md index 2e9d554..e423543 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -19,7 +19,7 @@ note's title-only chunk outrank a note whose body matches every query word. Full analysis on the epic. - [x] **Index the vault in a deterministic order and break BM25 ties on a stable key** ([#506](https://github.com/CryptoJones/omind/issues/506)) — **v10.4.1** — _bug_ -- [ ] **Recency must not lift a newer note's title-only match above a note whose body matches every query word** ([#507](https://github.com/CryptoJones/omind/issues/507)) — _bug_ — after #506 +- [x] **Recency must not lift a newer note's title-only match above a note whose body matches every query word** ([#507](https://github.com/CryptoJones/omind/issues/507)) — **v10.4.2** — _bug_ — after #506 ## Not planned @@ -75,6 +75,17 @@ _Closed as not planned or rejected. Kept for the record; nothing here is pending ## Done +### Shipped — 2026-10-31 (v10.4.2) + +- [x] **Recency must not lift a newer note's title-only match above a note whose body matches every query word** ([#507](https://github.com/CryptoJones/omind/issues/507)) — **v10.4.2** — _bug_ — + the recency boost is credited to a note's best matched content chunk (first in the keyword list, + else the vector list) instead of its `ord = 0` title chunk. Candidates stay the notes whose title + chunk a content leg returned; when any of them holds a chunk matching every query word, only those + receive the boost. (Epic decision 4's wider gate, and decision 7's no-gate fallback, each cost one + bench case on the reference vault; this rule ranks all 30 identically to 10.4.1.) Weights unchanged; + recency still never adds a note. `Hit.recency_rank` is filled in and `omind search --explain` + prints it. The strict `xfail` from v10.4.1 is removed. Last child of epic #505. + ### Shipped — 2026-10-31 (v10.4.1) - [x] **Index the vault in a deterministic order and break BM25 ties on a stable key** ([#506](https://github.com/CryptoJones/omind/issues/506)) — **v10.4.1** — _bug_ — diff --git a/CHANGELOG.md b/CHANGELOG.md index de6083c..f0de041 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [10.4.2] - 2026-10-31 + +### Fixed +- **A newer note that only shared the query's title could outrank an older + note whose body matched every query word + ([#507](https://github.com/CryptoJones/omind/issues/507), closes epic + [#505](https://github.com/CryptoJones/omind/issues/505)).** The recency leg + ranked notes newest-first but credited each note's `ord = 0` title chunk, + whatever chunk had actually matched. With two same-titled notes the decoy's + tied title match plus the recency boost beat the real answer's full-body match. + - The boost now goes to a note's best-ranked content chunk: its first chunk + in the keyword list, otherwise in the vector list. A note's keyword credit + and recency credit are no longer split across two chunks. + - Candidates are, as before, the notes whose *title* chunk a content leg + returned: recency breaks ties among notes that are about the query. When any + candidate also holds a chunk matching *every* query word, only those + candidates receive the boost. That is what drops the title-only decoy. + - Epic #505 decision 4 wanted every matched note admitted when the all-words + grade was empty. Measured on the reference vault that cost a bench case (a + day-old note at keyword rank 88 took the rank-1 boost: recall@5 80.0 → 76.7, + MRR 0.6616 → 0.6604), and so did decision 7's fallback (no gate at all). + The narrower rule above ranks all 30 cases identically to 10.4.1. + - Recency still only reorders: a note no content leg returned has no best + chunk and cannot be added. `_W_RECENCY` and `RRF_K` are unchanged. + - `omind search --explain` now reports `recency=N` beside the keyword and + semantic ranks; `Hit.recency_rank` was declared but never filled in. + - The strict `xfail` that 10.4.1 left on the decoy case is gone; the case + passes under both indexing orders. + ## [10.4.1] - 2026-10-31 ### Fixed diff --git a/docs/retrieval.md b/docs/retrieval.md index 6b76557..16019bf 100644 --- a/docs/retrieval.md +++ b/docs/retrieval.md @@ -37,7 +37,15 @@ Two rules keep the result honest: - **Recency re-ranks; it never adds.** A note that matched nothing cannot ride the recency leg into your results. (Before the index, search sorted purely by - date, so the newest notes came back regardless of relevance.) + date, so the newest notes came back regardless of relevance.) It is also + reserved for notes whose *title* the query matched — it breaks ties among + notes that are about the query, it does not reward a passing mention in a + long body. The boost is credited to the note's best *matched* chunk, not its + title chunk, and when any of those notes holds a chunk matching every query + word only such notes receive it — so a newer note that merely shares a title + cannot outrank an older note whose body answers the whole query. + `omind search --explain` shows the recency rank beside the keyword and + semantic ranks. - **Credential notes are de-prioritised** unless your query is itself about credentials — the same rule the consult gate applies. Search must never steer an agent toward the secrets notes. diff --git a/pyproject.toml b/pyproject.toml index 1d451df..838c5af 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "omind" -version = "10.4.1" +version = "10.4.2" description = "Reproduce the OMI/Obsidian memory integration for AI agents, plus a local web app to view, edit, and add memory entries." readme = "README.md" requires-python = ">=3.11" diff --git a/src/omind/__init__.py b/src/omind/__init__.py index ac1f1c0..0d7ee07 100644 --- a/src/omind/__init__.py +++ b/src/omind/__init__.py @@ -2,4 +2,4 @@ # Copyright 2026 Aaron K. Clark """omind — OMI/Obsidian memory tooling for AI agents.""" -__version__ = "10.4.1" +__version__ = "10.4.2" diff --git a/src/omind/cli.py b/src/omind/cli.py index e230130..0ab36ac 100644 --- a/src/omind/cli.py +++ b/src/omind/cli.py @@ -1414,7 +1414,8 @@ def _run_search_explain(omi_dir: Path, args: argparse.Namespace) -> int: where = f"#{hit.heading}" if hit.heading else " (title/name)" print( f"{rank}. {hit.filename}{where} score={hit.score:.5f} " - f"keyword={hit.keyword_rank or '-'} semantic={hit.vector_rank or '-'}" + f"keyword={hit.keyword_rank or '-'} semantic={hit.vector_rank or '-'} " + f"recency={hit.recency_rank or '-'}" ) # Usefulness (item #2): always show the reads behind the weight — both # the counted (organic) and filtered (hook/re-read) tallies — so a diff --git a/src/omind/searchindex.py b/src/omind/searchindex.py index 395834f..29d0798 100644 --- a/src/omind/searchindex.py +++ b/src/omind/searchindex.py @@ -1078,7 +1078,7 @@ def search( return self._listing(db, allowed, limit) scope = _scope(query, limit) expr = _fts_query(query) # the OR form; also snips the excerpts - keyword = self._keyword_leg(db, query, allowed, scope.depth) + keyword, all_words = self._keyword_leg(db, query, allowed, scope.depth) vector = self._vector_leg(db, query, allowed, scope.depth) matched = set(keyword) | set(vector) if not matched: @@ -1087,7 +1087,16 @@ def search( # note. Left unrestricted it made every query return the whole vault # newest-first — the exact failure this index exists to end. recency = [ - cid for cid in self._recency_leg(db, allowed, scope.depth) if cid in matched + cid + for cid in self._recency_leg( + db, + allowed, + scope.depth, + keyword=keyword, + vector=vector, + all_words=all_words, + ) + if cid in matched ] fused = _fuse( [(_W_KEYWORD, keyword), (_W_VECTOR, vector), (_W_RECENCY, recency)] @@ -1103,6 +1112,7 @@ def search( ranks = { "keyword": {cid: i + 1 for i, cid in enumerate(keyword)}, "vector": {cid: i + 1 for i, cid in enumerate(vector)}, + "recency": {cid: i + 1 for i, cid in enumerate(recency)}, } return self._materialize( db, query, expr, fused, ranks, scope.limit, scope.excerpt_chars, useful @@ -1133,11 +1143,16 @@ def _candidates( def _keyword_leg( self, db: sqlite3.Connection, query: str, allowed: set[str] | None, depth: int - ) -> list[int]: + ) -> tuple[list[int], set[int]]: """BM25 over the FTS index, graded: chunks matching every query word - first, then chunks matching any of them.""" + first, then chunks matching any of them. + + Returns the ranked chunk ids and, separately, the ids of the top grade + (every query word matched). The recency leg uses that set as its gate + (#507), so it is returned from here rather than computed twice.""" ranked: list[int] = [] seen: set[int] = set() + all_words: set[int] = set() for require_all in (True, False): expr = _fts_query(query, require_all=require_all) if not expr: @@ -1146,9 +1161,12 @@ def _keyword_leg( if chunk_id not in seen: seen.add(chunk_id) ranked.append(chunk_id) + if require_all: + all_words = set(ranked) if len(ranked) >= depth: break - return ranked[:depth] + head = ranked[:depth] + return head, all_words & set(head) @staticmethod def _bm25( @@ -1363,17 +1381,62 @@ def _weighting(self, db: sqlite3.Connection) -> _Weights: return weights def _recency_leg( - self, db: sqlite3.Connection, allowed: set[str] | None, depth: int + self, + db: sqlite3.Connection, + allowed: set[str] | None, + depth: int, + *, + keyword: list[int], + vector: list[int], + all_words: set[int], ) -> list[int]: + """Newest-first over the notes whose *name* the content legs matched, + credited to each note's best content chunk. + + Recency is a note-level signal, but RRF fuses chunks. The boost used to + land on a note's ``ord = 0`` title chunk whatever chunk had actually + matched, so a newer note whose *title alone* shared the query could + outrank an older note whose body matched every word (#507). Now: + + - a note is a candidate when its title chunk was returned by a content + leg, exactly as before: recency breaks ties among notes that are + *about* the query, it does not reward every passing mention. (Letting + every matched note in — epic #505 decision 4 — cost one bench case: + a day-old note at keyword rank 88 took the rank-1 boost.) + - when any candidate also holds a chunk matching *every* query word + (``all_words``), only those candidates receive the boost. That is what + drops the title-only decoy. + - the boost goes to the note's best-ranked content chunk — its first + chunk in the keyword list, otherwise in the vector list — so a note's + keyword credit and recency credit are no longer split across two + chunks. + - it still only reorders. A note no content leg returned has no best + chunk and cannot be added here. + """ + matched = set(keyword) | set(vector) + owners = self._owners(db) + best: dict[str, int] = {} + for chunk_id in (*keyword, *vector): + name = owners.get(chunk_id) + if name is not None: + best.setdefault(name, chunk_id) + if not best: + return [] rows = db.execute( "SELECT c.id AS id, c.filename AS filename FROM chunks c" " JOIN notes n ON n.filename = c.filename" " WHERE c.ord = 0 ORDER BY n.created DESC, n.filename LIMIT ?", (depth * 3,), ) - return [ - int(r["id"]) for r in rows if allowed is None or str(r["filename"]) in allowed + newest = [ + (int(r["id"]), str(r["filename"])) + for r in rows + if allowed is None or str(r["filename"]) in allowed ][:depth] + candidates = [name for title_id, name in newest if title_id in matched and name in best] + strong = {owners[cid] for cid in all_words if cid in owners} + graded = [name for name in candidates if name in strong] + return [best[name] for name in (graded or candidates)] def _weight_generated( self, db: sqlite3.Connection, fused: list[tuple[int, float]] @@ -1530,6 +1593,7 @@ def _materialize( agent=str(row["agent"] or ""), keyword_rank=ranks["keyword"].get(chunk_id, 0), vector_rank=ranks["vector"].get(chunk_id, 0), + recency_rank=ranks.get("recency", {}).get(chunk_id, 0), usefulness_weight=use.weight if use is not None else 1.0, useful_reads=use.useful_reads if use is not None else 0, filtered_reads=use.filtered_reads if use is not None else 0, diff --git a/tests/test_cli.py b/tests/test_cli.py index d0f7cd5..289e567 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -255,6 +255,25 @@ def test_search_finds_notes(tmp_path: Path, capsys: pytest.CaptureFixture[str]) assert "no matches" in capsys.readouterr().out +def test_search_explain_reports_the_recency_rank( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """#507: ``--explain`` names every leg that moved a result, recency included. + ``Hit.recency_rank`` existed but was never filled in, so the line showed only + keyword and semantic ranks.""" + from omind.store import NoteFields, OmiStore + + OmiStore(tmp_path / "OMI").create_note( + NoteFields(title="Codeberg Note", summary="about codeberg releases") + ) + args = ["search", "codeberg", "--explain", "--vault", str(tmp_path), "--folder", "OMI"] + assert main(args) == 0 + out = capsys.readouterr().out + assert "Codeberg Note" in out + assert "keyword=1" in out + assert "recency=1" in out + + def test_dedup_hint_warns_about_a_similar_existing_note( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] ) -> None: diff --git a/tests/test_name_timelines.py b/tests/test_name_timelines.py index 7c5955e..6a89f74 100644 --- a/tests/test_name_timelines.py +++ b/tests/test_name_timelines.py @@ -567,11 +567,11 @@ def test_preflight_names_resolve_through_recall( ), # #507: the Kp12 pair again, but the NEWER off-topic decoy's filename # sorts first. #506 breaks the title chunks' exact BM25 tie on filename, - # so the decoy's title chunk takes the tied keyword rank, and the - # recency boost (credited to a note's title chunk) carries it to the - # top, whichever order the vault was indexed in. Strict: this case must - # keep failing until #507 moves the boost onto the matched chunk. - pytest.param( + # so the decoy's title chunk takes the tied keyword rank; the recency + # boost used to land on that title chunk and carry the decoy to the + # top. Now the boost goes to A's matched Details chunk and the decoy, + # which matches no chunk on every query word, gets none. + ( [ ( "Kp12 deploy A.md", @@ -588,9 +588,6 @@ def test_preflight_names_resolve_through_recall( ], "Kp12 deploy steps ansible playbook frobnicate", "Kp12 deploy A.md", - marks=pytest.mark.xfail( - strict=True, reason="#507: recency lifts a newer title-only decoy" - ), ), ], ) diff --git a/tests/test_searchindex.py b/tests/test_searchindex.py index 9e4ba48..8419869 100644 --- a/tests/test_searchindex.py +++ b/tests/test_searchindex.py @@ -252,6 +252,74 @@ def test_ranking_is_the_same_whatever_order_the_vault_is_indexed_in( assert runs[0] == runs[1] +def _twin(omi: Path, filename: str, title: str, details: str, created: str) -> None: + (omi / filename).write_text( + f"# {title}\n\n## Metadata\n- Created: {created}\n- Tags: #ops\n\n" + f"## Summary\n{title}\n\n## Details\n{details}\n", + encoding="utf-8", + ) + + +def test_recency_credits_the_matched_chunk_and_skips_a_title_only_decoy(omi: Path) -> None: + """#507: two notes share a title. The older one's body matches every query + word; the newer one is about tomatoes. The recency boost used to land on + each note's title chunk, so the newer decoy's tied title match plus the + boost beat the real answer. The boost now goes to a note's best content + chunk, and only to notes holding a chunk that matched every query word.""" + _twin( + omi, + "Kp12 deploy A.md", + "Kp12 deploy steps", + "Kp12 deploy steps via ansible playbook frobnicate.", + "2026-09-01", + ) + # Filename sorts first, so with #506's tie-break its title chunk takes the + # tied keyword rank — the arrangement that made the old boost decisive. + _twin(omi, "Kp12 deploy 0.md", "Kp12 deploy steps", "unrelated garden tomatoes.", "2026-09-30") + for number in range(4): + _note(omi, f"Filler {number}", f"plain filler {number}", ["f"]) + idx = searchindex.SearchIndex(omi) + hits = idx.search("Kp12 deploy steps ansible playbook frobnicate") or [] + assert [h.filename for h in hits][:2] == ["Kp12 deploy A.md", "Kp12 deploy 0.md"] + winner, decoy = hits[0], hits[1] + # The boost was credited to the chunk that matched, not the title chunk… + assert winner.heading == "Details" + assert winner.recency_rank == 1 + assert winner.keyword_rank == 1 + # …and the decoy, which matches no chunk on every word, got no boost at all. + assert decoy.recency_rank == 0 + + +def test_recency_boosts_every_name_match_when_no_chunk_matches_all_words(omi: Path) -> None: + """#507: the all-words gate only applies when something passed it. Two + notes are each *named* for one word of a two-word query and no chunk holds + both; both are boosted, newest first, and ``recency_rank`` reports the order.""" + _note(omi, "Zebra", "zebra stripes", ["a"], created="2026-01-01") + _note(omi, "Giraffe", "giraffe spots", ["a"], created="2026-06-01") + _note(omi, "Other", "nothing relevant here", ["b"], created="2026-09-01") + idx = searchindex.SearchIndex(omi) + by_name = {h.filename: h for h in idx.search("zebra giraffe") or []} + assert set(by_name) == {"Zebra.md", "Giraffe.md"} + assert by_name["Giraffe.md"].recency_rank == 1 + assert by_name["Zebra.md"].recency_rank == 2 + + +def test_recency_ignores_a_note_the_query_names_nowhere_in_its_title(omi: Path) -> None: + """Recency breaks ties among notes that are *about* the query, as it always + did: a body-only mention earns no boost, and ``--explain`` shows a dash for + it rather than a made-up position. (Admitting every matched note cost a + bench case — a day-old note at keyword rank 88 took the rank-1 boost.)""" + _note(omi, "Target", "the specific thing", ["t"], created="2026-02-02") + _note(omi, "Specific", "the thing itself", ["t"], created="2020-01-01") + idx = searchindex.SearchIndex(omi) + by_name = {h.filename: h for h in idx.search("specific") or []} + assert set(by_name) == {"Target.md", "Specific.md"} + assert by_name["Target.md"].recency_rank == 0 # body-only: no boost + assert by_name["Specific.md"].recency_rank == 1 # named for the query + assert by_name["Specific.md"].keyword_rank == 1 + assert by_name["Target.md"].vector_rank == 0 + + def test_recency_only_reorders_matches_it_never_adds_notes(omi: Path) -> None: for number in range(5): _note(omi, f"Filler {number}", "unrelated filler", ["f"], created=f"2026-07-0{number + 1}") diff --git a/uv.lock b/uv.lock index 337a8a8..cc0d134 100644 --- a/uv.lock +++ b/uv.lock @@ -2210,7 +2210,7 @@ wheels = [ [[package]] name = "omind" -version = "10.4.1" +version = "10.4.2" source = { editable = "." } dependencies = [ { name = "certifi" },