Skip to content

[CI][Do not merge] Staging: automatic sharding on by default - #66965

Closed
Thangnguyenvn98 wants to merge 2 commits into
vllm-project:mainfrom
Thangnguyenvn98:agent/staging-shard-default-on
Closed

Thangnguyenvn98 wants to merge 2 commits into
vllm-project:mainfrom
Thangnguyenvn98:agent/staging-shard-default-on

Conversation

@Thangnguyenvn98

@Thangnguyenvn98 Thangnguyenvn98 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Overview

Throwaway staging PR for vllm-project/ci-infra#885 (automatic sharding on by default). Do not merge; it is closed once the scoped build is read.

Claims

  • With automatic_shard_default: true in ci_config.yaml and every automatic_shard: true deleted, pipeline generation succeeds.
  • A step and its AMD mirror shard with no flag; a fixed parallelism runs as it does today; a job that can't be sharded runs as one job and is listed in one warning annotation.

Validation

A scoped build with VLLM_CI_BRANCH=agent/shard-default-on on six keys: engine-1-gpu + its AMD mirror, language-models-tests-hybrid + its AMD mirror, distributed-torchrun-shutdown-tests-2-gpus, docker-build-metadata. Results follow as a comment.

Details

The change here is only the config line and the deleted flags. The generator change is in ci-infra#885.


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

@Thangnguyenvn98

Copy link
Copy Markdown
Contributor Author

Staging evidence for ci-infra#884, without merging it:

  • Control 102375: all four selected test shards and the hard coverage job passed, with no retries. The checker verified 797 outcomes. Independent node-ID comparison against main 102326 found 421 and 376 IDs for the two selected steps, with zero missing, added, outcome-changed or duplicated tests.
  • Deliberate missing-record experiment 102532: all four test shards passed and all 797 tests really ran, while the plant removed one recorded ID from each of four affected command records. The hard coverage job exited 1, failed the build, and named those four IDs. This is the intended negative result.

These experiments cover two selected engine steps and their mirrors, not every command shape. Deferred jobs run whole and unsharded. They used an earlier staged isolated-inventory path; the rebased #884 head is undergoing final review and a final control build. No universal rollout-safety claim is being made.

@Thangnguyenvn98
Thangnguyenvn98 force-pushed the agent/staging-shard-default-on branch 4 times, most recently from 646d60e to 5f269df Compare November 14, 2026 04:40
Thangnguyenvn98 and others added 2 commits November 14, 2026 01:33
Turns on automatic_shard_default in ci_config.yaml and deletes every
automatic_shard: true, to stage ci-infra#885 in a scoped build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
A step that opts out, a mirror that opts out alone, a mirror that opts in
while its step opts out, and pytest-shard flags in a job's PYTEST_ADDOPTS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
@Thangnguyenvn98
Thangnguyenvn98 force-pushed the agent/staging-shard-default-on branch from 5f269df to 3c86ffe Compare November 14, 2026 06:33
@Thangnguyenvn98

Copy link
Copy Markdown
Contributor Author

Build 102735 failed in bootstrap before any test shard started. The checkout and ci-infra revision were correct, but the installed generator omitted runtime_shard_coverage.py. A wheel-install reproduction failed identically; adding that module makes the installed-package check pass. I also added an isolated import check to ci-infra CI because the existing test suite puts the source checkout on Python’s path and hid the incomplete wheel. The final control will be rerun after checks and peer review.

@Thangnguyenvn98

Copy link
Copy Markdown
Contributor Author

Final validation for ci-infra#884 at c22b299e6f07027dfa7324f514b4f7791f278593, on vLLM 3c86ffe27a4ddf104aa6b628f0437eb8ff5dc743.

Control 102749 versus same-code main-generator baseline 102756:

  • Both runs checked out the exact vLLM commit and used two shards per selected step with identical test assignments. The control’s embedded plugin and checker matched the final branch byte for byte. No retries occurred in either run.
  • Selected step: 423 node IDs; 0 missing, added, changed or duplicated. Slowest shard 15.4 min versus 16.1 min; machine time 29.5 min versus 29.8 min (-1%).
  • Mirrored step: 378 node IDs; 0 missing, added, changed or duplicated. Slowest shard 12.5 min versus 12.4 min; machine time 22.2 min versus 22.5 min (-1%).
  • The hard coverage job verified 801 outcomes across the two steps and succeeded. That is exactly 423 + 378 IDs independently matched above.
  • The baseline used the main generator and had no coverage job. Every selected shard passed, and the slowest selected shard was 16.1 min.

Earlier planted-gap build 102532 deliberately removed four IDs from shard records: every shard passed and all 797 tests actually ran, but the hard coverage job failed the build and named all four IDs. Earlier packaging failure 102735 was corrected and covered by an installed-package CI check before the final control.

This is selected evidence, not universal rollout proof. Live cancellation, soft-fail, retried-shard, transport-failure and long-queue/deadline cases remain unexercised. Test IDs must be deterministic. Inventory collection has a 300-second bound; longer collection fails coverage rather than passing unchecked. A descendant that starts its own session can outlive the process group, so external resource cleanup is not guaranteed in that case. Both PRs remain unmerged.

@Thangnguyenvn98

Copy link
Copy Markdown
Contributor Author

Staging is complete. Final coverage evidence is recorded in #66965 (comment) and the ci-infra#884 description. This throwaway PR was never intended for merge.

@Thangnguyenvn98
Thangnguyenvn98 deleted the agent/staging-shard-default-on branch November 14, 2026 10:27
@github-project-automation github-project-automation Bot moved this to Done in NVIDIA Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant