Engine-version strata from the state-file history (#2) - #15
Merged
Conversation
Each ai-sync commit is stamped with engine_model/engine_tool_version in commits.jsonl, read from the document's state-file history rather than HEAD: the record written by the commit or its wave (first revision at-or-after, accepted within two days — exact for the same-commit and adjacent-commit modes), otherwise the record last in force; a sync predating the state file reads "unrecorded", and a state file's literal "model: unknown" keeps that value so the two cases stay distinguishable. run.json carries per-version totals (engine_strata: sync commits, prose and character churn, overwritten prose by prior tier, ordered by first appearance) and overwrites.json entries carry the same stamp. Verified on the real corpora: programming.zh-cn resolves a clean ladder from claude-sonnet-4-6@0.12.4 to claude-sonnet-5@0.26.0 with the April wave (22 sync commits at 0.14.1) isolated; python.zh-cn splits the July resync wave (claude-sonnet-5@0.17.0, 69 commits, 1,134 of the 1,249 human-overwritten lines) from the later versions and surfaces one literal model:unknown record. All other run.json quantities are byte-identical to the pre-strata scan, and the pinned intro.zh-cn regression stays green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds engine-version stratification to textstrata by resolving (model, tool-version) from each document’s .translate/state file history (not just the current HEAD), then publishing per-version aggregates in run.json.
Changes:
- Add state-file timeline parsing and
engine_version_at()to stampai-syncrows incommits.jsonlwithengine_model/engine_tool_version. - Aggregate sync activity into
run.json.engine_strataand stampoverwrites.jsonentries with the same engine identifiers. - Extend documentation and baseline tests to cover same-commit vs adjacent-commit state updates and aggregate ordering.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/textstrata/scan.py |
Implements state history timeline reading, per-sync engine stamping, and run-level engine_strata aggregation. |
tests/test_baseline.py |
Updates synthetic engine-era fixture state files and adds tests for engine stamping and engine_strata output. |
docs/method.md |
Documents the new engine-version strata method and updates known limits wording. |
CHANGELOG.md |
Notes the addition of engine-version strata and the new output fields. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…'s first sync The timeline (git log --follow plus a show per revision) was computed for every document up-front but consumed only by ai-sync rows; it is now read on the first sync row a document produces, so never-synced documents skip the git calls entirely. Behaviour is identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #2, the last open item on the critical path in #8: F1/F2 stratified by engine version, read from the
.translate/statehistory rather than just HEAD.What it does
ai-syncrow incommits.jsonlcarriesengine_modelandengine_tool_version, resolved from the document's state-file timeline. The rule: the record written by the commit or its wave — the first state revision at-or-after the commit, accepted within two days, which is exact for both engine commit modes (state-with-document, and the April wave's state-in-an-adjacent-commit one second later) — otherwise the record last in force. A sync predating the state file readsunrecorded; a state file that literally saysmodel: unknown(the case project-translation#22 found) keeps that value, so the two stay distinguishable.run.jsongainsengine_strata— sync commits, prose and character churn, and overwritten prose by prior tier per (model, tool-version), ordered by first appearance.overwrites.jsonentries carry the same stamp.docs/method.mddocuments the strata and tightens the pre-engine known-limits note.Verified
On the real corpora at the report HEADs: programming.zh-cn resolves a clean ten-rung ladder from
claude-sonnet-4-6@0.12.4toclaude-sonnet-5@0.26.0, with the April wave isolated as 22 sync commits at0.14.1; python.zh-cn splits the July resync wave (claude-sonnet-5@0.17.0, 69 sync commits, carrying 1,134 of the edition's 1,249 human-overwritten prose lines) from every later version, and surfaces one literalmodel: unknownrecord at0.20.0. Every otherrun.jsonquantity is identical to the pre-strata scan — the stamping is purely additive.Full suite green including the pinned intro.zh-cn regression; new unit and scan-level tests cover the same-commit, adjacent-commit, stale-state, pre-state and no-state-file cases on the synthetic engine-era fixture.
🤖 Generated with Claude Code