Repository navigation
feat(test): redesign CLI reporting and model usage summaries - #3285
Merged
Merged
Conversation
Render concurrent progress as a terminal tree with grouped final results and append-only CI output. Scope warnings to their execution context, distinguish model request retries from Case retries, and show concise failures with YAML locations and redacted inputs. Retain verbose diagnostics and structured artifacts, summarize actual models and token usage across attempts, and document the reporting behavior in English and Chinese.
Deploying midscene with
|
| Latest commit: |
f208b4b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://d8ac0844.midscene.pages.dev |
| Branch Preview URL: | https://feat-test-cli-reporting.midscene.pages.dev |
* fix(playground): isolate reports between executions * fix(core): recover report failures and isolate persisted dumps * test(computer): scroll timeline steps into view before AI assertion * fix(core): make resetDump reset the full report lifecycle Make resetDump asynchronous and prepare an independent report while reusing the Agent and device. Callers must await it before continuing execution. Group report state, separate memory cleanup from report transitions, seal artifacts with finalizeReport, and guard execution and cancellation boundaries. Unify Playground server and local adapters and document the API migration in English and Chinese. Validated 384 focused tests, root lint, Core/Playground/Report type checks, and the Report/Playground/Test build. * test(core): initialize agents in YAML regression fixtures Replace runYaml prototype calls bound to plain objects with initialized Agents in the Core docs and Web YAML validation tests. Keep model-facing actions mocked, disable report files, and dispose the Agents in finally blocks. Preserve validation of malformed YAML and task results, and verify that inline YAML does not replace the existing report filename. Validated complete Core and Web unit suites with coverage: 2,798 passed and 9 skipped. Core and Web type checks, root lint and git diff --check also passed. * style(site): fix sidebar configuration formatting Remove the blank line rejected by the non-fixing Biome check in CI. Verified pnpm run lint, pnpm exec biome check . --diagnostic-level=warn --no-errors-on-unmatched, and git diff --check. * fix(core): close report lifecycle gaps * docs(site): remove report lifecycle reference additions * refactor(core): extract agent report management and option normalization
* feat(test): add file concurrency with maxConcurrentFiles * fix(test): classify shared resource failures by explicit ownership * docs(site): clarify file concurrency and resource lifecycle * feat(test)!: combine project and file concurrency limits Use one file scheduler with independent file and live Project limits. Default omitted limits to 1 and allow null to remove either limit. Update lifecycle tests and bilingual documentation. BREAKING CHANGE: replace test.maxConcurrency with the explicit maxConcurrentProjects and maxConcurrentFiles settings. * docs(site): clarify setup, fileSetup and beforeAll * fix(test): tailor concurrency migration to project scope --------- Co-authored-by: quanru <quanruzhuoxiu@gmail.com>
Render concurrent progress as a terminal tree with grouped final results and append-only CI output. Scope warnings to their execution context, distinguish model request retries from Case retries, and show concise failures with YAML locations and redacted inputs. Retain verbose diagnostics and structured artifacts, summarize actual models and token usage across attempts, and document the reporting behavior in English and Chinese.
yuyutaotao
approved these changes
Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
midscene-testcurrently mixes repeated Step progress, retries, warnings, report paths, and failures in the same log stream. This change adds a concise default reporter with a concurrent live tree in interactive terminals and append-only output in CI.--reporter verbose.Validation
pnpm run lintpnpm exec nx run @midscene/shared:build --excludeTaskDependenciespnpm exec nx run @midscene/core:build --excludeTaskDependenciespnpm exec nx run @midscene/test:build --excludeTaskDependenciespnpm run type-check:testspnpm exec tsc -p packages/test/tsconfig.json --noEmit --pretty falsepnpm exec nx run @midscene/test:test --excludeTaskDependenciespnpm --filter @midscene/test exec vitest --run --config vitest.e2e.config.ts tests/e2e/create-cli.test.tspnpm --filter @midscene/cli test:e2e:shared-browserLint passes with one existing unused-suppression warning in
apps/site/theme/components/TestFramework.tsx. The Midscene Test unit suite passes all 541 tests across 50 files. All 29 generated-project integration tests and all 6 shared-browser CLI E2E tests pass. Model-backed demos have not been rerun for the CI fix.