Repository navigation
Release v0.12.0: P2 benchmark batch 2 and failure-path fixes - #16
Merged
Merged
Conversation
Adds live HN/Wikipedia tasks, independent URL+DOM verifiers, the shared dual-host runner, offline policy tests, and sanitized first-batch evidence. Failed trials are retained without retry or budget relaxation. Fixes #15
Track settled per-ask receipts, distinguish unrun and unknown usage from confirmed zero, and keep unknown token totals through failure summaries. Add offline regressions and recompute the preserved live batch without rerunning sites. Refs #15
Handle ego spawn errors without losing earlier host results, clean partial staging allocations, and propagate CDP partial cleanup problems. Add four isolated offline fault regressions with failing-before evidence. Refs #15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P2 第二批:共同受控基准扩至 12 项,并提供真实 Jev 的 HN / Wikipedia 双宿主测量入口。
新增
p2-controlled-v1历史清单,新增十二项p2-controlled-v2,覆盖缺文本来源与两字段批处理,细分预期拒绝与预算统计。bench/live-benchmark/使用相同目标、输入、预算和独立 URL + DOM 验证器,显式--live才联网。null与原因。修复
验证与复跑
node bench/test-release-consistency.mjs:20/20。node bench/test-live-benchmark.mjs、node bench/test-common-benchmark.mjs:exit 0。node bench/live-benchmark/run.mjs --live --hosts both --tasks all --rounds 3。十二次试验全部完成,10/12 达成、错误成功 0、清理失败 0,模型jev-1.14.0。HN 两宿主各 3/3;Wikipedia CDP 1/3、ego 3/3。两次 CDP 失败均为五步预算耗尽,命令如实 exit 1,样本完整保留。原始数据与脚本哈希:测量说明。后续异常路径修复未改写原始测量数据或重复站点测量。
已知边界
真实测量仅 macOS arm64、两任务每宿主各三轮,不能外推普遍成功率或稳定宿主性能差异。Windows/Linux、远程或用户提供的 CDP endpoint 未验证。费用和人工接管仍为
null;MCP 继续为实验性入口。更新:
ego-decision-layer update;旧版无 update 子命令的用户按原安装渠道更新。完整变更:CHANGELOG.md。