feat: chat-export transcripts, idempotent harvest, linear splitter - #157
Closed
titanwings wants to merge 3 commits into
Closed
titanwings wants to merge 3 commits into
titanwings wants to merge 3 commits into
Conversation
added 3 commits
September 11, 2026 06:00
A person's own conversations are the best material for distilling them, and ChatGPT, Claude, Telegram, Slack, and Discord all export JSON. Those files were pretty-printed as generic JSON, which sorts the mapping tree and loses who said what. The application/json parser now recognizes those exports structurally and renders one verbatim line per message with a normalized time and speaker, keeps the user's standing instructions, reports every omission as a warning, and leaves any other JSON document byte-identical to the previous rendering.
… people An independent audit of the subject-resolution work found that harvesting the same folder twice stored each file again as new evidence, that a name living in another space was silently duplicated into the default space, that a cross-space pair of same-named people was resolved by the create conflict instead of being shown, and that a differently capitalized name silently split one person in two. Harvest now resolves the name before it creates anything: one match is reused, several are listed for the operator to choose from, and none warns when a near-duplicate exists. Files whose bytes this subject already stored are skipped with --force to override, recorded in a local harvest-state.json.
…arvest An independent audit of the byte-exact splitter found a hang: a long run of combining marks made the cut back-off walk back to the start of an empty part forever, so harvesting such a file never returned. The same back-off scanned the whole remaining line for every part, which was quadratic on one huge line. The splitter now canonicalizes first (CRLF to LF, NFC, trailing spaces removed), measures each line once, bounds every cut adjustment, and never leaves a part empty, so it terminates, runs in linear time, and cannot hand the engine a part that normalization would reject or rewrite. Harvest also keys its record on the real path, merges the record when another run wrote it, applies --limit after the already-ingested filter, and reports a file it cannot read instead of letting a raw error escape.
Owner
Author
|
Superseded: this stacked series is consolidated into the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
feat: chat-export transcripts, idempotent harvest, linear splitter
Commits
Stack
Stacked on
pr/07-subject-show-and-split; the whole series ends atdistilly-work(0bd924e).Verification
Every feature carries an author report and an independent audit in the delivery evidence folder (
host-verification/), including screenshots and the audit verdict that drove the fixes. Gates on the top of the stack: format, lint, docs, release check, and 1163 tests.Real hosts
Codex, Claude Code 2.1.268, OpenClaw 2026.9.2, Hermes v0.19.0, and DSH 0.1.5-rc.1 were each exercised end to end (install, host-visible confirmation, exactly five MCP tools, person Skill install/remove, uninstall). Capacity fixtures are recorded for Codex 0.146.0, OpenClaw 2026.3.24, and Hermes v0.9.0; newer host versions run on the labelled conservative floor.
Evidence
Author reports and screenshots (delivery evidence folder
host-verification/):Independent audit reports: