[Shayne Boyer (spboyer)]

Summary

Adds protected exact-unit checkpoint creation and recovery to the source Lab. The panel retains original failures/ordinals, distinguishes unknown progress from zero, and downloads the unchanged protected response. Installed v0.40.1 artifacts are unchanged.

Stack: #942 → #943 → #958 → #959 → this PR. Current head 72323b625a2bc48498e78bb046141fda2e9bab4b, tree ebf7e361689b51de01f04d95f937bc456c5146d0; base successor/765-core-public at d3312fad538ce449c7ad13aec37df12979eefe07. The one needed dependency integration consumes capture 5c4fcc47 through the core. Normal merges preserve both histories; no main refresh or rewrite.

Related issue

Related to #764 and #765. Neither issue is complete or intended to close.

Agent handoff

• Scope: explicit create/resume, original/retry/reuse display, strict declaration correlation, exact protected downloads, producer-backed browser controls and current documentation/screenshot.
• Key files: web/src/api/checkpointResults.ts, web/src/lab/Checkpoint*.tsx, internal/webapi/checkpoint_browser_fixture_test.go, checkpoint browser spec/fixtures and affected guides.
• Decisions: no raw checkpoint GET; feat: recover protected evidence capture libraries (partial #768) #958 remains capture-policy owner. A private host-only checkpointOperations.observers forwards existing hooks without weakening concrete backend admission. Eleven fixtures come from real registered HTTP handlers and native codecs; error fixtures use actual admitted concrete MockEngine workspace failure. Canonical declaration hashes match fixed native typed field order, UTF-8 sorting and Go JSON escaping, including independently produced HTML/Unicode/non-BMP cases. Hash linkage is not storage or execution authority.
• Gaps: three feat: captured checkpoint CLI and HTTP source preview #959 threads cover two unresolved infrastructure-result classes (close failures and pre-store failure) requiring an approved frozen-wire decision. Generic protected Submit, ordered evaluation-wide lineage, native mid-dispatch kill, Windows/full-platform acceptance and independent Terra approval remain outstanding. Browser cancellation is not worker-termination evidence. No issue closure or release claim.

Type of change

• Bug fix
• New feature
• Documentation update
• Refactor or maintenance
• CI/CD or release change

Validation

Linux amd64, execution/vet Go 1.27.3, Node 26.12.1. Latest local source is frozen as 72323b62/tree ebf7e361; current native producer and focused browser controls also executed after that commit. Earlier bounded capture/public-consumer gates at e5283dcf, clean-source fixture gates at 0772eda1, and browser CI at 90ca587d remain explicitly historical. The new bounded selection/policy fix intentionally changes dashboard/site source and its matching built entrypoint.

• Latest source tree frozen as 72323b62: dashboard npm run lint, npm run build, full npx --no-install playwright test --project=chromium --retries=0 --workers=2 PASS 544 / 1 skipped, 5.8m; all 59 checkpoint controls execute. The sole existing skip requires WAZA_CALIBRATION_REPORT_DIR for independent Calibrate/Verify exports; no calibrated producer evidence is claimed. Site npm run build PASS 32 static controls/build and npm run test:e2e -- --project=chromium --retries=0 PASS 54/54. Current checkpoint screenshot regenerated identically; unrelated generated screenshots restored only to their verified clean pre-run bytes. Tracked dashboard index references the actual physical matching build, without staging ignored bundles.
• Latest source: guarded NO_COLOR=1 GOMAXPROCS=2 go test -race -p 2 -count=1 -run '^TestCheckpoint' ./internal/webapi ./internal/webserver PASS 5.048s/1.180s; affected API vet and canonical checksum-pinned schema/lint PASS, 0 issues; runtime marker absent. Exact committed 72323b62 additionally runs the native HTTP producer and 14 focused Chromium regressions, PASS 12.5s, marker absent. No full Go/compatibility/native Windows rerun is claimed.
• Review regressions RED before fix: two single-field decision-selection attempt mismatches plus three consent-only requests wrongly render and offer download; all five fail the new controls. Rich native reordered-array positive passes before and after the fix. Native ForRequest proves consent-only identity/admission equality but policy mismatch, actual HTTP 409/source-withheld/null-progress/no new calls; all three negotiation arrays reversed retain compatibility and the original failed terminal. Null/omitted policy fields are rejected by native codecs and browser correlation; no defaulted consent/arrays. Existing ten fixture entries and protected response bytes are retained exactly; one new real HTTP policy fixture is added.
• Exact e5283dcf: guarded NO_COLOR=1 GOMAXPROCS=2 go test -race -p 2 -count=1 ./internal/evidencestate ./internal/ownedsubmission PASS 14.638s/3.557s; NO_COLOR=1 GOMAXPROCS=2 go test -race -p 2 -count=1 -run '^TestCheckpoint' ./cmd/waza ./internal/webapi ./internal/webserver PASS 26.256s/5.035s/1.297s. Affected five-package GOMAXPROCS=2 go vet -p 2 and canonical checksum-pinned schema/lint PASS, 0 issues. Verified runtime marker absent. Covers current protected consumers, producer-backed fixtures, configured-store/production negatives and real compiled fresh-child recovery. Does not execute or waive the known large-request finalized-publication timeout in internal/runnercheckpoint.
• go test ./...: no complete passing Lab-head claim. Parent 2d4b37db full uncached guarded NO_COLOR=1 go test -p 2 -count=1 ./... failed grade cleanup permission, embedded CLI installation and snapshot same-inode/after-verification-read controls. Parent native Windows run 42901719302 failed capture-owned TestFrozenCanonicalAndPrecision; not waived.
• Final 0772eda1 canonical Go lint: PATH=<session-tools>:$PATH GOTOOLCHAIN=go1.26.4 GOMAXPROCS=2 RUNNER_TEMP=<validation> bash .github/scripts/lint.sh passed checksum-pinned schema and lint, 0 issues; GOMAXPROCS=2 go vet -p 2 ./cmd/waza passed. Earlier package-load timeout remains recorded, not passing evidence.
• Previous c08bc870 guarded NO_COLOR=1 GOMAXPROCS=2 go test -race -p 2 -count=1 ./internal/webapi ./internal/webserver passed 52.041s/46.621s. NO_COLOR=1 GOMAXPROCS=2 go test -race -count=1 ./cmd/waza -run '^TestCheckpoint(PublicCLI|ProductionProcessRecovery)' passed 22.680s; GOMAXPROCS=2 go vet -p 2 ./... passed. Includes actual HTTP denied reuse/original-source byte preservation and actual compiled flag, OS stdin-read and native JSON-admission errors: exit 6, sanitized JSON, private operator cause and no storage creation. Live opt-ins disabled; verified rejecting runtime marker absent.
• Final 0772eda1: guarded NO_COLOR=1 GOMAXPROCS=2 go test -race -p 2 -count=1 -v -run '^(TestCheckpointSourceSnapshotIncludesDirtyAndIgnoredInputs|TestCheckpointProductionProcessRecovery)$' ./cmd/waza passed 25.623s, marker absent. Actual current native inputs from go list -mod=readonly include changed/new source and ignored embedded assets; byte/hash/inventory checks precede a test-owned genuine Git commit and normal build. No clean-caller prerequisite/HEAD-only snapshot/shared-root staging/fake revision. Tiny dirty/deleted/untracked/ignored controls prove compiled output and unchanged caller Git status. One ordinary owning CLI (567 actual inputs) serves all recovery cases: no-kill, pre-kill persisted-state verification, syscall-stop/SIGKILL/reap/join/fresh-child and every-original-store-file preservation. Dispatched-without-completion remains actual service producer + compiled consumer, not CLI killed mid-dispatch. Linux only.
• Docs at 90ca587d: cd site && npm run build passed 32 static controls and build. npm run test:e2e -- e2e/runner-terminal.spec.ts --project=chromium --retries=0 passed 3/3 with deployed-base links and actual screenshot. Earlier full Chromium was 53 passed / 1 timeout in the 120-second full-link inventory; not a full-suite pass.
• Web at 90ca587d: npm run lint, npm run build passed. npx --no-install playwright test e2e/checkpoint-operations.spec.ts e2e/screenshots.spec.ts --project=chromium --retries=0 --workers=2 passed 56/56 (44 checkpoint + 12 screenshot controls). Earlier local full run had 502 passed / 2 comparison-refresh failures / 1 skipped; baseline attribution was not established.
• Manual validation: docs/site checkpoint images identical; tracked index references matching physical ignored bundles; only task-owned unrelated generated screenshots restored. No dependencies, bundles or scratch files staged.
• Not applicable

Actual frozen HTTP producers cover success, failed checks, failed-original finalized reuse, fresh before-unit retry, execution error and its finalized reuse, unavailable/incompatible/stale source, post-completion publication failure, and Unicode canonicalization. Tests assert every rendered value and exact downloaded bytes. Mutations demonstrate rejection of raw reports, retagging, caller/decision/context contradictions, bad original exits/call counts, completion IDs/digests, fabricated progress, unknown/duplicate error graders and transport contradictions; valid partial/empty error verdicts remain available.

Dashboard workflow PR triggers target main only. Manual run https://github.com/microsoft/waza/actions/runs/42917223110 at 90ca587d passed 529 / 1 skipped; actual logs prove all missing-source, partial-publication and reverse-action controls executed. Subsequent CLI-only merge run https://github.com/microsoft/waza/actions/runs/42918476608 at 0eb85d19 completed successfully; neither run targets final 0772eda1. Current-head full/native/platform acceptance remains unestablished. No absent/skipped job is called green.

Documentation

• README updated
• Site guides updated
• Producer-backed screenshot and private captured-request guidance updated
• Not applicable

Risk and rollback

• Risk: Medium. Source-preview execution UI; explicit actions may incur charges only under existing captured-request consent.
• Rollback: revert Lab commits without deleting immutable source/attempt/checkpoint records or removing feat: captured checkpoint CLI and HTTP source preview #959.

Notes for reviewers

Received review bodies and inline findings have been read; verified fixes are replied in their original threads. Canonical completion, decision, fresh identity and requested-source binding fixes are included. The latter was reproduced red with an unchanged same-work reused response sent for another real boundary digest, and a canonical timestamp change; both are now refused before display/download. Independently reordered keys remain valid for both before-unit retry and finalized reuse, with exact altered-byte downloads. Independent parent Terra approval and complete required current-head checks are still required; author execution and automated review summaries do not substitute for them.

Current 72323b62 fixes review 5850810577 findings 4531621718 and 4531482232 within permission 6781714437: the decision's complete selection matches current admission, without retagging older original reservation/terminal attempts; compatible source policy matches native versions/required/optional/paid ordering, sorting and Go JSON encoding. Unavailable/incompatible/source-withheld results retain their explicit failure state. No native codec, runtime, capture policy, core head or aggregate registration/component is changed.

Final action-context regressions also reproduced red using unchanged actual protected producers: create results replayed as source-less completed resumes (including completed work with outer publication failure), and finalized reuse replayed as create. All are now refused before render/download, while legitimate source-withheld failures remain available. No frozen 1.0 profile change.

No runtime service.go, provider, capture-policy or Report JSON changes. The screenshot records protected mock API evidence, not inference or synthetic-demo execution. Both PRs remain draft; no merge, release or approval bypass.

[Copilot]

🟡 Changes recommended

Admission checks accept contradictory terminal exit codes and call counts while rejecting valid error results with partial or empty verdicts, compromising displayed and exported results and hiding checkpoint downloads.

2 open issues

• Validate terminal exit codes and call counts against submissions · New
• Allow incomplete grader verdicts for error observations · New

What changed in this PR

Adds checkpoint creation and recovery to the source Lab using the existing protected HTTP API. This extends the partial #765 preview without changing installed v0.40.1 artifacts or completing evaluation-wide lineage and platform acceptance.

Changes:

• Adds create/resume controls, original-versus-new execution details, and unchanged protected JSON downloads.
• Adds response validation, producer-backed fixtures, and browser coverage.
• Updates documentation and the dashboard entrypoint for the source preview.

File	Description
web/​src/​lab/​RunnerNegotiation.tsx	Integrates the checkpoint panel.
web/​src/​lab/​CheckpointResult.tsx	Displays recovery status, lineage, counts, and diagnostics.
web/​src/​lab/​CheckpointOperations.tsx	Adds create/resume controls and protected downloads.
web/​src/​api/​runner.ts	Exports shared selection and identity comparisons.
web/​src/​api/​checkpointResults.ts	Validates protected results and sends checkpoint requests.
web/​e2e/​fixtures/​checkpoint-operations.json	Adds seven recorded protected HTTP fixtures.
web/​e2e/​checkpoint-operations.spec.ts	Covers rendering, downloads, rejection, and stale-request handling.
web/​dist/​index.html	References the updated dashboard bundle.
site/​src/​content/​docs/​guides/​immutable-artifacts.mdx	Documents Lab recovery operations and limits.
site/​src/​content/​docs/​guides/​dashboard-lab.mdx	Explains checkpoint controls and recorded screenshot evidence.
site/​e2e/​runner-terminal.spec.ts	Checks updated guidance and deployed screenshot loading.
README.md	Announces source Lab checkpoint support.
internal/​webapi/​checkpoint_operations.go	Forwards host-only hooks for producer controls.
internal/​webapi/​checkpoint_browser_fixture_test.go	Generates and checks fixtures through protected HTTP handlers.
docs/​PRD.md	Records partial checkpoint delivery and remaining scope.
docs/​EXECUTION_CHECKPOINTS.md	Updates Lab behavior and recovery limitations.
docs/​DEMO-GUIDE.md	Adds the checkpoint demonstration workflow.
CHANGELOG.md	Records protected API/Lab support and outstanding acceptance.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Completion linkage mismatches can be displayed and downloaded, while valid error observations with partial or empty verdicts are rejected, hiding their checkpoint references and downloads.

1 open issue

• Validate completion linkage for every submitted result · New

2 resolved since last review

• Validate terminal exit codes and call counts against submissions
• Allow incomplete grader verdicts for error observations

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Critical validation gaps allow mismatched execution IDs, altered reservation identities, and inconsistent attempt IDs to pass checks and be displayed or exported as valid results.

3 open issues

• Bind decision execution ID to the caller and reservation · New
• Require full identity equality for fresh submissions · New
• Validate completion linkage for every submitted result

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Resume responses are not bound to the requested checkpoint digest, allowing a response for a different checkpoint of the same work to be accepted, displayed, and downloaded.

1 open issue

• Bind resume responses to the requested digest · New

3 resolved since last review

• Require full identity equality for fresh submissions
• Bind decision execution ID to the caller and reservation
• Validate completion linkage for every submitted result

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The returned checkpoint source is not validated against the requested digest, allowing a mismatched response to pass admission and expose a different checkpoint for display or download.

1 open issue

• Bind resume responses to the requested digest

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Human assessment is needed because an omitted critical finding shows that resume results can bypass digest validation and be displayed or downloaded without being linked to the requested checkpoint.

No open issues 🎉

1 resolved since last review

• Bind resume responses to the requested digest

_{Review effort: Balanced}

[Shayne Boyer (spboyer)]

Waza release triage: Lab recovery now carries #765's existing v0.41.0 label/milestone and dashboard/evaluation/CLI area labels. It remains a partial draft.

The green dashboard run does not cover the failing required Go path. Direct inspection of Linux run 42931822028 at 72323b625a2bc48498e78bb046141fda2e9bab4b shows TestCheckpointProductionProcessRecovery/stdin-admission-diagnostics: expected invalid JSON object key, actual invalid JSON object. This is the same diagnostic mismatch as parent #959, not a demonstrated new Lab rendering failure.

Next action: consume only the parent owner's evidenced correction/disposition in dependency order, then refresh applicable exact-head checks. Retain the summary-only lineage/policy admission concerns and actual error-envelope/source acceptance requirements; zero open inline threads is not completed review. This triage makes no source or stack changes and does not declare Lab/release parity.

[Copilot]

🔵 Needs a closer look

Unresolved infrastructure-result contracts and incomplete current-head native-platform validation across the recovery stack require independent contract review and acceptance evidence.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Protected recovery spans multiple contracts, with unresolved infrastructure-result wire decisions and unestablished current-head Go/compatibility and native Windows acceptance requiring independent integration review.

No open issues 🎉

_{Review effort: Balanced}

[Shayne Boyer (spboyer)]

Independent GPT-5.6 Terra review: no significant issues found; merge-ready for review. The PR body’s stated platform and feature follow-ups remain open; this is not full #765 acceptance.

[Shayne Boyer (spboyer)]

Waiting for #764. At the exact head 7048b1b6c537a0169d6ffe59d0ae7e0434e779a4, all nine reported PR checks are green, but #764 is still open and #765 explicitly depends on the logical-attempt model. The latest #765 issue note says the merged #943 checkpoint core is partial and asks that this issue remain open behind #764; #961’s own handoff also says platform/public CLI/API/Lab and full acceptance remain incomplete. Green CI and a review comment do not satisfy those criteria, and no waza41:verify pass is recorded.

Shayne Boyer (@spboyer), please complete and land the #764 prerequisite (including its remaining review/contract gates); then resume #765 against the stable dependency and finish the outstanding production-CLI/public integration, Lab, documentation, and verifier acceptance. No source change or duplicate PR was made in this run.

[Copilot]

🟡 Changes recommended

An in-flight parse can commit stale results or errors after the input changes or a newer parse starts; generation guards and a pending-parse/edit regression test are needed.

1 open issue

• Prevent stale async parses from overwriting newer results · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Human assessment is needed because an unpublished finding indicates that routine Playwright runs rewrite tracked documentation images, with both browser projects targeting the same files.

No open issues 🎉

1 resolved since last review

• Prevent stale async parses from overwriting newer results

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Protected recovery spans native, capture, HTTP, and browser contracts, with unresolved infrastructure-result wire decisions and native-platform acceptance requiring independent owner review and validation.

No open issues 🎉

_{Review effort: Balanced}