[BUG] update_memory silently mis-resolves the entities field on ~5% of edits (text field unaffected) — no error, deterministic, not fixed by retry

[cvhowlett]

Summary

update_memory silently mis-resolves the entities field on a small but consistent fraction of edits, while the text field is always written correctly. The wrong entity is not random noise — it's an existing, valid entity already in the bank, sometimes a completely unrelated real person, so the result reads as a normal, well-formed record. There is no error, no partial-failure indicator, and 200 OK is returned on the corrupted writes exactly as on the correct ones.

Environment

• Package: hindsight-all 0.9.0 (self-hosted, pip install, embedded Postgres backend)
• Called via the Hindsight MCP server, from Claude Code
• macOS 25.3.0 (Darwin arm64)

Repro

1. Have a bank with several hundred existing memories and an entity graph that includes near-duplicate/related names (e.g. FirstName / FirstName LastName, or multiple people who share a surname).
2. Run a bulk update_memory correction across many records — in this case, 657 calls each replacing a generic placeholder entity (User) with the correct person's real name in a record's entities list (and updating text to match).
3. After the batch, re-fetch every edited record (list_memories or per-id fetch) and diff the actual stored entities array against the entities you submitted for that call.

Observed

• text field: 657/657 correct. No mismatches anywhere.
• entities field: 25/523 entity-bearing edits (~4.8%) came back different from what was submitted, and none of the 25 are cosmetic — e.g. (names/labels below are placeholders standing in for real values, structure and specificity preserved):
  • Submitted ["topic A", "Person B", "Person C", "Person C Full Name"] → stored ["topic A", "Person B", "Person C", "Person D Full Name"] — an existing, unrelated real person (Person D, who shares a surname with the intended target) was swapped in on a high-value financial fact.
  • Submitted ["Medication X", "Person C Full Name"] (a medication name + a person) → stored ["Person E", "Person C Full Name"] — the medication name silently became a different existing entity, an unrelated person, even though nothing about this edit touched that entity.
  • Submitted ["Person C Full Name", "Account Label 1", "topic"] → stored ["Person C Full Name", "Account Label 2", "topic"] — an untouched account-name entity was swapped for a different existing account entity belonging to the same person.
  • An entity not targeted by the edit at all (present unchanged in both old and new lists) came back as a different real contact already in the bank.
  • 11 of the 25 simply reverted the entities field to the pre-edit value despite text updating correctly and the new entities being submitted.
• Deterministic, not a race condition. A single isolated sequential retry (zero concurrency) reproduced the exact same wrong output. A batch of 8 parallel retries reproduced the identical wrong output 8/8 times. Same input → same wrong output, every time.
• Not fixable by resubmission, for 24 of the 25 — retrying with the same or a cleaned-up entity list changed nothing.
• One partial mechanism found: 1 of the 25 had a near-duplicate short/long-form pair in the submitted list (e.g. "Person C" and "Person C Full Name" together). Dropping the short form and resubmitting the long form alone fixed that one record. This does not explain the other 24 — several of those have no duplicate-form pair anywhere in the submitted list (e.g. the medication and account-label cases above).
• Clusters by source document, not randomly distributed. Grouping the 25 mismatches by the source document the memory was originally extracted from: 17 of 25 trace back to just 4 source documents (5, 5, 4, and 3 mismatches respectively), with 8 further isolated singletons across 8 other documents. Fields failing within the same source document fail in unrelated ways (different entities, different substitution targets) — it isn't one bad value propagating, it looks scoped to something about the document/chunk rather than the specific entity string.

Expected

Either the stored entities field matches what was submitted, or the call fails/reports a conflict — not a silent, confidently-wrong substitution returned as a successful write.

Why this matters more than a normal bug

text was correct on every single edit, so a spot-check of a few records looks completely clean. The only way this surfaced was a deliberate full post-hoc diff (re-fetch all 657 edited records, compare entities arrays field-by-field against what was sent). Without that audit, a sensitive financial fact would have sat silently attached to the wrong real person (an actual family member of the intended person, already an entity in the bank) indefinitely — discoverable only if someone later asked a reflect/graph-scoped question about that wrong person and got an answer partly built from a fact that was never theirs.

What I haven't verified

• Root cause is unconfirmed — the document-clustering pattern suggests something cache- or index-scoped in the entity-resolution path of update_memory's write, but this is inference from black-box I/O only; I didn't have access to server internals/logs.
• Not tested against the Docker distribution, only the pip (hindsight-all) self-hosted install with the embedded Postgres backend — unknown whether this is install-specific.
• Not tested against a fresh/small bank — this bank had ~2,000 pre-existing facts and an entity graph with several genuinely similar/overlapping names, which may or may not be a precondition.

Data

I have the full input/output diff for all 25 mismatched records (record id, expected entities, actual entities) and the complete 657-record batch that was submitted. The real values are personal data, so I've kept this report to structurally-accurate placeholders — happy to provide an anonymized version of the diff (ids + expected/actual shape, values replaced with consistent placeholders) if that would help debugging.

[cvhowlett]

Follow-up from the original report: a clean single-edit reproduction (no bulk operation involved), plus two new observations and a workaround that works. Names are placeholders as before, structurally matched to the real data.

Setup

Same bank as the original report. Extraction had created two entities for the same person from a spelling variation in the source notes: Dr. Waller (correct) and Dr Wall (typo). Four world facts referenced the typo entity. Goal: repoint their entities to the correct one via update_memory.

Reproduction (single edits, ~30s apart)

1. Edit 1 — fact A (document X): sent entities: ["Dr. Waller", "CareOrg", "Craig H"] → stored ["Dr. Waller", "CareOrg", "Craig H"]. Correct.
2. Edit 2 — fact B (same document X): sent entities: ["Dr. Waller", "CareOrg", "Parent"] → stored ["Dr Wall", "CareOrg", "Parent"]. The supplied Dr. Waller was silently re-resolved to the typo entity. 200 OK, well-formed response, no warning; the other two entities and a simultaneous text change applied correctly.
3. Edit 3 — fact C (different document Y): same shape → stored ["Dr Wall", ...] again.
4. Edit 4 — immediate identical retry of edit 3 → stored ["Dr Wall", ...] again. Sticky: retrying never helps, consistent with the original report's "not fixable by resubmission."

So once a typo entity exists, update_memory cannot be used to move facts onto the correctly spelled entity — the edit-time resolver keeps snapping the corrected name back to the typo node, silently.

New observations vs the original report

• Not bulk-specific. The original report involved 657 bulk edits; this is one careful edit at a time, with verification between each. Same corruption.
• Not strictly document-clustered. Edits 1 and 2 were on facts from the same source document, one minute apart, with near-identical payloads — one resolved correctly, one didn't. The document-clustering we reported originally appears to be a tendency, not a rule.

Workaround that works

retain → verify → invalidate:

1. retain new content stating the facts with the correct name (the LLM extraction path resolved Dr. Waller correctly in every attempt today — the failure appears confined to the edit-time resolver).
2. Verify the new units' entities before touching anything.
3. invalidate_memory the old records (reversible, keeps the audit trail).

Happy to run diagnostics on this bank if useful — the broken state is preserved via the invalidated records.