Skip to content

[rel/4.5] Fix packaged MTP Test Explorer discovery and execution (#11993) - #12054

Merged
Amaury Levé (Evangelink) merged 1 commit into
rel/4.5from
dev/amauryleve/4-5-test-explorer-backport
Oct 20, 2026
Merged

Amaury Levé (Evangelink) merged 1 commit into
rel/4.5from
dev/amauryleve/4-5-test-explorer-backport

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Backport #11993, Fix packaged MTP discovery and execution in Visual Studio Test Explorer, to rel/4.5.

Test Explorer selects the .build.appxrecipe as the test source and otherwise tries to execute that XML file, producing Win32 error 193. This generates source-adjacent launcher configurations that select the existing project-local packaged-app controller and explicitly preserve --server. The controller accepts bare/explicit JSON-RPC server arguments and the client port without weakening native SDK transport validation.

Caller-owned/edited configurations, incremental generation, clean tracking, design-time builds, explicit opt-outs, and native run-command overrides retain the source fix's behavior. This does not fix packaged-project debugging.

Release scope and provenance

  • Source merge: a7d7711ee4f843a0245b18ef3f3bdedfc9d1264a.
  • Validated backport head: 2d7fec599ed4e12853560745f1b9317551204fbb.
  • Tested release baseline: a223d46d5784d315e7bae2a04d85185b191dd2b3. Before publication, the live release tip was refreshed to 851ffb1578b10fb7c79e6e96245a662481a43e56; its intervening changes are three unrelated server-client unit-test files, with no shipping-source/build-input changes. git merge-tree --write-tree origin/rel/4.5 HEAD succeeds and the PR diff remains exactly the original four files.
  • Existing packaged controller/native SDK support already ships on this release. No SDK, target framework, dependency, or unrelated feature change is imported.
  • The original patch applies without conflicts. Preserve the release's existing net9/net10 prefix-based controller selection and its omission of the main-only EnableMicrosoftTestingPlatform fixture property.

The release-pinned SDK is 12.0.100-alpha.1.26480.103. Fresh MSTest 4.5.2-dev and Microsoft.Testing.Platform 2.5.2-dev packages were built in this worktree. All six relevant package nuspecs identify the validated head, both net8.0/net9.0 Windows controller payloads are complete, and the supplemental composed consumer's six cached nupkg files match the fresh Shipping package bytes and source paths exactly. No installed 2.5.1 payload or sibling worktree's 2.5.2-dev cache supplied that consumer.

Microsoft.Testing.Extensions.PackagedApp.MSBuild.2.5.2-dev.nupkg SHA-256:
6CF20CE32F01CD5AB91C119164DF0FFE3C0200183814AEF8C8A7D31819A42AAB.

Executed validation

Use the repository-local toolchain bootstrapped by the release build scripts. A short process-only PATH was necessary because the inherited PATH prevented cmd.exe children from locating PowerShell. Build/consumer caches are worktree-local; the acceptance assembly additionally creates its own fresh effective package cache for every run.

$env:Path = "$PWD\.dotnet;$env:SystemRoot\System32\WindowsPowerShell\v1.0;$env:SystemRoot\System32;$env:SystemRoot;C:\Program Files\Git\cmd;C:\Program Files\PowerShell\7"
$env:DOTNET_ROOT = Join-Path $PWD '.dotnet'
$env:NUGET_PACKAGES = Join-Path $PWD 'artifacts\.nuget\build'
$env:NUGET_HTTP_CACHE_PATH = Join-Path $PWD 'artifacts\.nuget\http'
$env:DOTNET_PROCESSOR_COUNT = '2'
$env:MSBUILDDISABLENODEREUSE = '1'
$env:MSBUILDUSESERVER = '0'

The full source-pack attempt generated the release packages, but its final repository-wide inventory audit failed, as detailed below. The two changed shipping products were then packed explicitly:

# Exit 0.
.\.dotnet\dotnet.exe pack src\Platform\Microsoft.Testing.Extensions.PackagedApp\Microsoft.Testing.Extensions.PackagedApp.csproj --no-restore --configuration Debug -p:SkipUapTargetFramework=true -m:2 -nodeReuse:false -bl:artifacts\log\backport-targeted-pack-extension.binlog --verbosity quiet

# Exit 0. Includes both shipping Windows controller TFMs.
.\.dotnet\dotnet.exe pack src\Platform\Microsoft.Testing.Extensions.PackagedApp.MSBuild\Microsoft.Testing.Extensions.PackagedApp.MSBuild.csproj --no-restore --configuration Debug -p:SkipUapTargetFramework=true -m:2 -nodeReuse:false -bl:artifacts\log\backport-targeted-pack-msbuild.binlog --verbosity quiet

# Exit 0; 0 warnings, 0 errors.
.\.dotnet\dotnet.exe build test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj --framework net9.0 -p:SkipUapTargetFramework=true -m:2 -nodeReuse:false -bl:artifacts\log\backport-unit-build.binlog --verbosity quiet

# Exit 0; 0 warnings, 0 errors.
.\.dotnet\dotnet.exe build test\IntegrationTests\MSTest.Acceptance.IntegrationTests\MSTest.Acceptance.IntegrationTests.csproj --framework net10.0 -p:NetCurrent=net10.0 -p:SkipUapTargetFramework=true -m:2 -nodeReuse:false -bl:artifacts\log\backport-acceptance-build.binlog --verbosity quiet

# Exit 0: 189 total, 187 passed, 0 failed, 2 skipped.
.\.dotnet\dotnet.exe artifacts\bin\Microsoft.Testing.Extensions.UnitTests\Debug\net9.0\Microsoft.Testing.Extensions.UnitTests.dll --filter FullyQualifiedName~PackagedApp --report-trx --report-trx-filename packaged-unit.trx --results-directory "$PWD\artifacts\TestResults\green-unit"

# Exit 0: 85 total, 85 passed, 0 failed, 0 skipped; 126 child processes.
# The invocation used NUGET_HTTP_CACHE_PATH=artifacts\.nuget\acceptance-http and DOTNET_ROLL_FORWARD=Major.
.\.dotnet\dotnet.exe artifacts\bin\MSTest.Acceptance.IntegrationTests\Debug\net10.0\MSTest.Acceptance.IntegrationTests.dll --filter FullyQualifiedName~PackagedAppIntegrationTests --report-trx --report-trx-filename packaged-acceptance.trx --results-directory "$PWD\artifacts\TestResults\green-acceptance"

The two unit skips are intentionally non-Windows-only cases, not skipped regression proof. All nine new Test Explorer parser cases passed. All twelve retained TestExplorer_ acceptance cases executed and passed, including caller edits followed by build/clean/rebuild, opt-outs, and the apostrophe/ampersand intermediate-path case. The full 85-case class also executed native overrides, incompatible/missing payload handling, shipping-controller success/failure/empty results, timeout cancellation, and multi-target exit-code aggregation.

Test-value evidence at the validated head

Contract Counterfactual / preserved behavior Executed result
Bare/explicit JSON-RPC server flags, client port, stale native transport Retained new parser cases cover inline/bootstrap arguments and uppercase JSON-RPC; existing malformed/duplicate native transport guards remain selected. No paired baseline parser run is claimed here. All 9 new cases passed within the 187 executed unit cases.
Executable/recipe launcher generation and lifecycle Replace only the integration nupkg's buildTransitive targets with exact a223d46d5784d315e7bae2a04d85185b191dd2b3 targets; retain the current controller and tests. Both generation rows fail with missing .launcher.config.json, exit 2. Restore the original nupkg byte-for-byte and rerun the same rows. Red: 0 passed/2 failed. Green: 2 passed/0 failed, exit 0. This is a single-target package counterfactual, not a full baseline rebuild.
Fresh shipping launcher/controller/host composition An isolated net8.0 consumer loads the generated recipe launcher and starts its actual packaged controller. Honor the passive controller's multipleConnectionProvider=true handshake, then initialize its separate test-host connection and issue discover/run requests. Exercise both --server and --server jsonrpc. Each mode: protocol 1.0.0, MTP 2.5.2-dev on both connections, 2 discovered, 2 passed/0 failed, two application-written marker files, controller exit 0. Forced-loose activation, not a Visual Studio/registered-app test.
Existing process/transport behavior Retained package-consuming tests cover normal/failure/zero-result/cancellation/override/multi-module paths. Entire class: 85 passed, 0 failed, 0 skipped.

The retained counterfactual selection was:

.\.dotnet\dotnet.exe artifacts\bin\MSTest.Acceptance.IntegrationTests\Debug\net10.0\MSTest.Acceptance.IntegrationTests.dll --filter FullyQualifiedName~TestExplorer_GeneratesIncrementalExecutableAndRecipeLaunchers --report-trx --report-trx-filename red.trx --results-directory $counterfactualOutput
# With baseline package targets: exit 2, both rows fail with DirectoryNotFoundException for PackagedConsumer.launcher.config.json.

.\.dotnet\dotnet.exe artifacts\bin\MSTest.Acceptance.IntegrationTests\Debug\net10.0\MSTest.Acceptance.IntegrationTests.dll --filter FullyQualifiedName~TestExplorer_GeneratesIncrementalExecutableAndRecipeLaunchers --report-trx --report-trx-filename green.trx --results-directory $counterfactualOutput
# With the original package restored: exit 0, both rows pass.

The baseline targets were extracted with git archive and copied into the nupkg through ZipArchive; the package was restored in finally. The restored SHA-256 is the original 6CF20CE32F01CD5AB91C119164DF0FFE3C0200183814AEF8C8A7D31819A42AAB.

Supplemental shipping-protocol consumer build:

# NUGET_PACKAGES=artifacts\.nuget\protocol-consumer; NUGET_HTTP_CACHE_PATH=artifacts\.nuget\protocol-http.
# Exit 0; 0 warnings/errors. This scratch consumer/harness is not added to the four-file source diff.
.\.dotnet\dotnet.exe build artifacts\test-explorer-smoke\ProtocolConsumer.csproj --configuration Debug -m:2 -nodeReuse:false -bl:artifacts\log\backport-protocol-consumer-build-02.binlog --verbosity quiet

The supplemental harness reads ProtocolConsumer.build.launcher.config.json, adds only the client host/ephemeral port (and jsonrpc for the explicit variant), initializes both advertised connections, and sends testing/discoverTests, testing/runTests, and exit. It checks exact test counts, application marker contents, and parent exit status. This is additional composed-product evidence, not a substitute for the retained acceptance regressions.

Limits and failed broader check

No discoverable Visual Studio installation was available (vswhere -latest -products '*' -format json returned []), so real Test Explorer and registered WinUI/UWP activation were not exercised. No packaged-debugging fix, full acceptance-suite run, or CI result is claimed.

The final full-source pack attempt used:

$env:DOTNET_ROLL_FORWARD = 'Major'
& "$env:SystemRoot\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -ExecutionPolicy Bypass -Command '& .\eng\build.ps1 -restore -build -pack -msbuildEngine dotnet -nodeReuse:$false -properties @("/p:Projects=$PWD\src\**\*.csproj", "/p:SkipUapTargetFramework=true", "/m:2", "/bl:artifacts\log\backport-pack-07.binlog"); exit $LASTEXITCODE'

Exit 1: the final eng\verify-nupkgs.ps1 inventory check expected classic UAP assets deliberately omitted by SkipUapTargetFramework=true: MSTest 10/9 expected/actual files, TestAdapter 76/64, TestFramework 105/88. No source or audit-gate change was made to hide this. The affected modern-.NET controller packages and the actual consumers described above were validated separately; classic UAP packaging/full-inventory validation remains unavailable here.

Earlier environment/bootstrap attempts also exposed the oversized PATH and the source-glob analyzer-generator build-order issue; neither required a product change. A first single-connection scratch protocol client stalled because it ignored the passive controller's advertised capability; the successful supplemental runs use the correct two-connection contract.

Local artifacts are retained: artifacts\log\backport-*.binlog/transcripts, artifacts\log\backport-package-provenance.json, both artifacts\TestResults\green-*\*.trx, and session counterfactual/protocol transcripts, marker files, and JSON summaries. They are not uploaded/public CI artifacts; the commands, exact counts, hashes, and result excerpts above are the reviewer-facing evidence.

Backport #11993 to rel/4.5.

(cherry picked from commit a7d7711)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 900409d0-52c8-4c88-8eb2-fecde4a51897
@github-actions github-actions Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026
@Evangelink
Amaury Levé (Evangelink) merged commit d9833cd into rel/4.5 Oct 20, 2026
19 of 24 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/4-5-test-explorer-backport branch October 20, 2026 18:38
@github-actions github-actions Bot removed the state/needs-review Awaiting review from the team. label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants