Skip to content

Profile pipeline & write integrity: parse crashes, blanking writes, version races (3 PRs) #22

Description

@ahueb

Three verified defects across the profile generation pipeline and every writer that touches ResearcherProfile.

These three must ship together, in order: V1 → V6 → C1. They edit overlapping code: C1 rewrites profile.profile_version = (x or 0) + 1 at profile_pipeline.py:338 and agent_page.py:1048, and those lines sit inside the blocks V6 rewrites (profile_pipeline.py:331-339 — the whole if synthesized: body; agent_page.py:1042-1047 — the six field assignments immediately above). Splitting them across PRs in different issues guarantees a merge conflict.

Line numbers current as of origin/main @ b7edcbc (2026-07-30).

PR V1 — ORCID/PubMed/pipeline null-&-type safety + publication dedup (small-medium)

  • COR-15orcid.py:127 summary.get("external-ids", {}).get("external-id", []) runs outside the try (the try at :103-109 wraps only the HTTP fetch); ORCID returns "external-ids": null for DOI/PMID-less works → AttributeError: 'NoneType' object has no attribute 'get' → the step-3 catch zeroes the whole profile. The same pattern is one line up: summary.get("title", {}).get("title", {}).get("value", "") (:113) crashes identically on ORCID's "title": null. Both reproduced 2026-07-30. Fix: null-safe parse at both sites.
  • COR-16 — no (user_id, pmid) unique constraint (models/publication.py declares no __table_args__; only a non-unique index exists) → duplicate rows. pubmed.py:198/204 read .text, which truncates at the first child element (repro: "Role of <i>TP53</i>…""Role of "). _validate_profile (profile_pipeline.py:445) does research_summary.split() on an LLM nullAttributeError (reproduced); the techniques check uses len() not isinstance (:453). Fix: a (user_id, pmid) unique constraint + ORCID-group dedup; itertext(); null/type-safe validation.

PR V6 — Profile-write integrity + atomic writes (medium)

  • COR-22run_profile_pipeline overwrites research_summary/techniques unconditionally under if synthesized: (not validated, profile_pipeline.py:331); pending_profile is read (profile.py:71, admin.py:87) but never written anywhere in src/; save_public_profile reads every field from Form("") and overwrites unconditionally (agent_page.py:1019-1047) so a partial POST blanks the profile. Fix: wire or delete pending_profile; gate the store on validated; don't overwrite fields absent from a POST.
  • COR-24 (write-safety slice) — no os.replace/tempfile/flock; profile_export.py:118/142 truncate-then-write via path.write_text; public writers order DB→disk while save_private_profile orders disk→DB (agent_page.py:917 vs 1050/1060) → disk (what the agent reads) and DB (what the web reads) can diverge permanently; create_revision writes a full snapshot with no content-dedup (unbounded TEXT growth). Fix: atomic temp+rename writes; consistent ordering; content-dedup the revision write. (Consolidating all writers into one save service is deliberately deferred — this PR only makes the existing writers safe.)
  • COR-23 — the pipeline writes private_profile_seed to the DB but never calls export_private_profile (profile_export.py:126; the only callers are onboarding.py:285 and agent_page.py), and the agent reads disk only → runs on "No private instructions yet." until the PI completes that POST. Fix: export the seed at generation time.

PR C1 — Atomic RMW for the two proven races (small)

ResearcherProfile.profile_version = (x or 0)+1 at 4 sites (profile_pipeline.py:338, onboarding.py:154, agent_page.py:1048, profile.py:167) and the delegate_slack_ids ARRAY whole-column reassign (agent_page.py:1189-1192,1375-1378, invite.py:241-244) are read-modify-write across await points — the original audit reproduced lost updates 200/200 on both. Fix: atomic SQL (= col + 1, array_append/array_remove) or a version_id_col.

Definition of done: each PR ships a test that fails against the pre-fix code, including a migration test for the new unique constraint against a table that already contains duplicates.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions