Skip to content

prompt-refiner: Step 1 ("read context") has no manuscript/research-project awareness #265

Description

@Data-Wise

Gap

prompt-refiner's Step 1 ("Read context") is scoped to generic software-project signals:

detect project type (DESCRIPTION / package.json / pyproject.toml), current git branch, and
.STATUS current-task if present.

There's no notion of a research/manuscript project — no check for a bibliography
(references.bib), an existing notation glossary (a "Table 1"-style symbol table), prior review
docs (docs/reviews/), or a companion R package. For manuscript-revision requests, this means
the raw "context" step returns almost nothing useful, and the actual grounding work (grepping the
manuscript for a term, checking a citation against references.bib, reading a prior review doc)
has to happen ad hoc, outside the skill's own procedure, before a genuinely well-grounded refined
prompt can be written.

Evidence

From a Missing Effect (research manuscript) session, 2026-07-05: multiple /craft:workflow:refine
calls needed manual pre-refine investigation the skill's Step 1 doesn't prompt for — e.g. grepping
index.qmd for a flagged term's actual usage count before refining "check my terminology," or
checking docs/reviews/ for an existing report before refining "generate a report... save to
internal docs" (to avoid proposing a redundant save location). In each case the refined prompt
ended up naming concrete files/counts/locations found this way — but only because the investigation
happened manually, not because Step 1 surfaced it.

Proposed fix

Extend Step 1's context-detection with a manuscript/research-project branch, parallel to the
existing software-project branch:

  • Detect a manuscript project by presence of references.bib + a .qmd/.tex main file (already
    a pattern savant's own project-detection logic uses, e.g. in /savant:restore).
  • If detected, additionally note: whether a notation/symbol glossary table exists in the main
    file, whether docs/reviews/ (or an equivalent existing-reports folder) has prior review docs
    on the same topic, and whether a .flow/research-config.yml exists (journal/register context).
  • This is read-only, matching Step 1's existing "read-only" constraint — just a wider set of
    files to check for a different project archetype.

Why this isn't just "the model already does this"

It's true a capable agent can do this investigation unprompted — and in practice this session's
agent did. But the skill's own documented procedure doesn't ask for it, so it's inconsistent
across sessions/models whether that grounding happens before the refined prompt is shown, versus
being skipped in favor of a plausible-sounding but ungrounded rewrite. Encoding it in Step 1 makes
it a reliable step rather than an emergent behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions