Skip to content

Prevent competing Graph recovery workers - #71

Merged
gateway merged 3 commits into
mainfrom
fix/graph-recovery-ownership
Oct 24, 2026
Merged

gateway merged 3 commits into
mainfrom
fix/graph-recovery-ownership

Conversation

@gateway

@gateway gateway commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Repeated recovery requests could prepare or schedule a run while its worker was active. GraphRuntime now reserves the run before preparation/scheduling and releases ownership on completion or failure. Preserve cancellation and legitimate orphan/restart recovery within the supported single API process.

recover/start/direct execution
  reserve run ownership
  prepare and execute once
  release in finally

CI selects the official Ubuntu archive through the existing runner mirror list. Signing keys, suites, packages, every gate and the30-minute timeout remain intact; verify ffmpeg/ffprobe execute.

Evidence

  • Before: guarded whole-runtime reproduction shows competing workers; earlier CI stalled on Azure package downloads before tests.
  • After: reviewed f847daf, merged bca2da6; entire trees match. All11 exact-head PR and all11 exact-main checks passed. Each full suite passed1,074 backend tests including all12new actual recovery cases,1,427 web tests, lint/types/static/build and hosted browser smoke.
  • Corrected the initially invalid new fixture using actual model.image → image_refs → preview.image connections, retaining all ownership/cancellation/failure assertions and adding input/result propagation checks. No validation bypass.
  • Both independent review rounds: Standards0/Spec0.
  • Actual production-built candidate and postmerge Codex Browser passed recovered history/status/results: Recovered,13completednodes,18storedartifacts/outputimages. Browser starts no recovery; hosted router/runtime/store tests exercise it.
  • Original browser stores/tabs and14protected/config tables/.env/safeguards preserved. No local pytest/collection or paid generation. Evidence/bundles hash-verified before nonforce own-worktree cleanup; branch/history retained. Final finding closed after cleanup; five-fix batch complete.

Merge Danger

Door: two-way

Blast Radius: Graph recovery

No schema/provider change or distributed ownership guarantee. Stop predecessor before successor orphan recovery.

@gateway
gateway marked this pull request as ready for review October 24, 2026 01:56
@gateway
gateway merged commit bca2da6 into main Oct 24, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant