Skip to content

Prose-gate unpaired additions and deletions - #16

Merged
mmcky merged 1 commit into
mainfrom
prose-gate-unpaired
Aug 28, 2026
Merged

Prose-gate unpaired additions and deletions#16
mmcky merged 1 commit into
mainfrom
prose-gate-unpaired

Conversation

@mmcky

@mmcky mmcky commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Closes #14 — the dominant pair-precision cause measured by the three-edition adjudication (QuantEcon/project-translation#53).

An unpaired line with no target-script content — code, maths, markup, or .translate frontmatter arriving or leaving — now routes to code-or-markup instead of addition/deletion, mirroring what categorise() already does for non-prose paired lines. The omission taxonomy therefore counts prose only, which is what the adjudication showed it was failing to do (addition precision 4/13 / 8/18 / 1/20 across intro / python / programming, the misses almost entirely code and metadata rows).

As #14 recommended, non-prose lines are re-routed rather than dropped, so the total pair count is unchanged — the regression test pins the corpus pair count against the 2026-08-04 study, and a category-only shift keeps it green by construction (verified: full suite passes including the pinned regression).

Real-corpus effect at programming.zh-cn @ b727528, the pathological case: addition 103 → 30, deletion 80 → 38, code-or-markup 25 → 140, total 255 unchanged; composition and engine_strata byte-identical. The direction and scale match the adjudication sample, where 19 of 20 sampled additions were non-prose.

The synthetic fixture gains one purely additive and one purely deletive editor-shaped commit, so the routing is asserted without depending on git's hunk alignment or the similarity pairing (which cross-pairs freely inside mixed hunks — the separate, smaller artefact #14 leaves out of scope). docs/method.md's line-pairing known-limit now states the rule.

🤖 Generated with Claude Code

An unpaired line with no target-script content — code, maths, markup or
metadata arriving or leaving — now routes to code-or-markup instead of
addition/deletion, mirroring what categorise() already does for non-prose
paired lines, so the omission taxonomy counts prose only. The total pair
count is unchanged (categories shift only), which keeps the regression's
pinned corpus pair count green by construction.

At programming.zh-cn @ b727528: additions 103 -> 30, deletions 80 -> 38,
code-or-markup 25 -> 140, total 255 unchanged; composition and engine
strata byte-identical. The fixture gains one purely additive and one purely
deletive editor-shaped commit so the routing is asserted without depending
on hunk pairing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 28, 2026 02:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a precision defect in the omission taxonomy by applying the same “prose gate” used for paired lines to unpaired additions/deletions during scanning, so that non-prose (code/maths/markup/metadata) lines no longer inflate addition/deletion.

Changes:

  • Route unpaired added/deleted lines to code-or-markup when they contain no target-script characters, otherwise keep addition/deletion.
  • Add a synthetic baseline test that asserts the new routing for a purely-additive and purely-deletive edit shape.
  • Update method documentation and the changelog to describe the rule and its intended effect (category-only shift; total pair count unchanged).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/textstrata/scan.py Prose-gates unpaired additions/deletions and assigns taxonomy via CATEGORY_MAP to keep non-prose edits out of omission buckets.
tests/test_baseline.py Extends the synthetic repo fixture and adds a focused test asserting unpaired-line routing and taxonomy for code lines vs prose.
docs/method.md Documents the known-limit rule that unpaired non-prose lines are categorised as code-or-markup.
CHANGELOG.md Records the behavioural change and clarifies that totals are preserved (only categories shift).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@mmcky
mmcky merged commit f8ce40d into main Aug 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unpaired additions and deletions are not prose-gated, inflating the omission bucket

2 participants