Upgrade vendor SDKs, including Codex 0.144.4 - #56
Draft
ebarti wants to merge 10 commits into
Draft
Conversation
Owner
Author
|
Final verification update for head
The draft PR is mergeable and current with |
ebarti
changed the title
## Summary - records the reproduced false-green and missing-baseline failure modes on current `main` - defines the evidence invariants, First Principles model, Five Whys, and owning fix layers - lays out a five-PR delivery stack with exact files, tests, acceptance criteria, and report-only rehearsals - keeps the replacement stack independent of stale open PRs #19 and #40 ## Why The Agent SDK stages execute successfully, but the documented evolution workflow can label ambient import failures as locked SDK evidence, collapse incomplete probes into a passing headline, and permanently fail CLI-binary snapshots through the wrong module name. The repair needs to preserve the explicit candidate-inspection and credential-scrubbing boundary while making the supported command actionable. ## Stack This is PR 1 of 5 and contains planning documentation only. The implementation branches will be based in order on this branch: 1. truthful locked-baseline collection 2. fail-closed evidence summaries and exact transition gates 3. Codex CLI binary introspection 4. actionable runtime-specific operator commands and documentation conformance ## Validation - `git diff --cached --check` - reviewed against `origin/main` at `2e7e4c3` - checked against the current implementation, tests, documentation, and live PR topology
## Summary - treat `uv.lock` as the authoritative SDK baseline during explicit candidate inspection - isolate locked baselines when the matching SDK is absent or the ambient environment has drifted - keep non-opt-in runs honest by recording the actual ambient version instead of stamping it as locked - convert isolated API snapshot setup, install, execution, timeout, and malformed-output failures into bounded evidence ## Stack - Depends on #51 - This is layer 2 of 5 in the SDK evolution demo reliability stack ## Review boundary This layer repairs baseline provenance and snapshot collection only. Behavior-probe subprocess failures and the final `fail > incomplete > changed > pass` assessment are intentionally implemented together in the next layer so incomplete evidence cannot temporarily appear green. ## Validation - `PYTHONPATH=. .venv/bin/python -m pytest -q` — 421 passed, 12 skipped - `.venv/bin/ruff check .` - `.venv/bin/mypy` - `uv lock --check` - `git diff --check`
## Summary - parse resolver updates once as exact package/from/to transitions and reuse them across collection and deterministic gates - derive canonical locked-baseline expectations from collected package evidence, including no-update runs - contain isolated behavior setup, install, execution, timeout, and malformed-output failures as bounded structured evidence - compute `fail > incomplete > changed > pass` from raw probes, diffs, full probe pairing, and trusted expectations - recompute the canonical assessment for reports and implementation gates instead of trusting a cached headline - persist `behavior_summary.json`, include it in the current-state manifest, and surface snapshot/behavior errors and reasons in the Markdown report - require exact API-diff transitions and fail closed on missing, malformed, contradictory, incomplete, or breaking behavior evidence ## Stack - Depends on #52 - This is layer 3 of 5 in the SDK evolution demo reliability stack ## Review notes The layer received a two-pass correctness review. Review findings covered drifted no-update baselines, contradictory cached summaries, malformed nested details, failure precedence, partial probe sets, empty self-declared expectation scope, and real breaking-payload coverage; all were repaired and re-reviewed with no remaining actionable findings. ## Validation - `PYTHONPATH=. .venv/bin/python -m pytest -q` — 449 passed, 12 skipped - focused SDK evolution suite — 90 passed - `.venv/bin/ruff check .` - `.venv/bin/mypy` - `uv lock --check` - `git diff --check` Security-diff scans were intentionally not part of this phase review.
## Summary - map the `openai-codex-cli-bin` distribution to its real `codex_cli_bin` import module - use that module consistently for current and isolated API snapshots - require metadata, module import, `bundled_codex_path()`, and an existing regular file before the binary probe passes - preserve the `binary-distribution` probe identity for every in-process and embedded failure path - keep success evidence stable without persisting temporary executable paths - serialize binary-probe failures with bounded exception context while redacting everything from the first absolute POSIX, drive, UNC, or URL path start ## Stack - Depends on #53 - This is layer 4 of 5 in the SDK evolution demo reliability stack ## Review notes Two correctness passes covered wrong-module snapshots, metadata/import/helper/conversion/file-check failures, invalid helper returns, regular-file semantics, embedded probe labels, cross-platform path forms, diagnostic bounds, and path-bearing messages with spaces. No actionable findings remain. ## Validation - `PYTHONPATH=. .venv/bin/python -m pytest -q` — 477 passed, 8 skipped - SDK evolution test file — 114 passed - Codex CLI focused selection — 23 passed - real installed locked Codex wheel snapshot/probe — passed - `.venv/bin/ruff check .` - `.venv/bin/mypy` - `git diff --check` Security-diff scans were intentionally not part of this phase review.
## Summary - make every supported operator command use a locked uv environment and the matching runtime extra - require explicit candidate inspection for actionable report and implementation passes - document deterministic behavior-summary handoff and stop conditions - add structural command-conformance tests that reject bare, compound, commented, echoed, or runtime/extra-mismatched examples ## Stack This is PR 5 of 5 and is based on agent/sdk-evolution-cli-bin-snapshots. ## Validation - uv lock --check - uv run --locked ruff check . - uv run --locked mypy - uv run --locked pytest -q: 491 passed, 8 skipped - focused operator-doc conformance: 14 passed - ordinary correctness review: clean after two repair passes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrade every monitored vendor runtime to the current audited release:
claude-agent-sdk:0.2.106->0.2.128openai-codex:0.1.0b3->0.144.4openai-codex-cli-bin:0.137.0a4->0.144.4google-antigravity:0.1.4->0.1.8This branch is reconciled with
mainatd649ffb57bb1588a5b452f125b47870af979d24e, after the SDK-evolution workflow stack landed.The repository-wide eight-day freshness delay remains in place for ordinary dependencies. The four monitored SDK/runtime packages are explicitly exempted in the audited evolution workflow and project lock policy, and the Codex SDK range is widened to admit the current
0.144.xline.Evolution evidence
Report-only completed before implementation from a clean transition baseline using the locked Codex extra, supported local Codex authentication, explicit candidate inspection, and package-scoped freshness bypasses.
pass(8snapshots,0import/execution errors)4changed000falsefalsetruepass, with no required changesObserved compatible drift:
CanUseToolShadowedWarningandModelUsage, and changesResultMessage.CodexConfig; required thread start/run parameters remain present.LiteRTAgentConfig,LiteRTBackend,LocalOpenAIAgentConfig,proto, andutils, and changesAgentConfig/LocalAgentConfigwhile preserving required fields.No adapter redesign was required. Vendor-specific additions remain explicit rather than being flattened into the common API.
Implementation and verification
The implementation stage repeated the evidence and reviewer gates, then applied the four-package lock refresh.
env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE uv lock --checkenv -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE uv run --locked ruff check .env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE uv run --locked mypyenv -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE uv run --locked pytest492 passed, 8 skippedcodex-cli 0.144.4.Safety and uncertainty
0.144.4was found. The reviewer classified this as explicit non-blocking uncertainty because the candidate adapter and bundled-binary contracts pass.