Skip to content

Add opt-in CI jobs without replacing normal selection - #906

Open
taneem-ibrahim wants to merge 4 commits into
vllm-project:mainfrom
taneem-ibrahim:ci-include-jobs
Open

taneem-ibrahim wants to merge 4 commits into
vllm-project:mainfrom
taneem-ibrahim:ci-include-jobs

Conversation

@taneem-ibrahim

@taneem-ibrahim taneem-ibrahim commented Nov 12, 2026 •

Copy link
Copy Markdown

Purpose

Support vLLM's /ci run --include, so PR authors can add selected optional jobs without enabling the full nightly set. VLLM_CI_INCLUDE_STEP_KEYS reuses key validation and dependency expansion while preserving normal CI selection. Deploy this support before merging the vLLM PR.

The existing retry-only selector replaces normal selection; Buildkite's Run more jobs needs Buildkite access and an active build. This enables the additive GitHub command without widening permissions.

Unknown or off-platform keys warn and discard the include request, preserving normal CI and the docs-only skip. Existing rerun restrictions remain unchanged. The shared parser is renamed _parse_step_keys; three existing line reflows in buildkite_step.py are required by ruff-format.

Reproducer / main vs branch

Dry-run current vLLM job definitions with the three optional pooling keys:

main: requested jobs with include setting = 0
branch: requested jobs with include setting = 3
branch: normal jobs preserved = True

Test Plan

.venv/bin/python -m pytest buildkite/tests/pipeline_generator -q --ignore=buildkite/tests/pipeline_generator/test_fnrec.py: 402 passed. The excluded recorder suite has the same eight setup failures on unchanged main. Scoped pre-commit passed; local-only checks cover invalid-key fallback, dependencies, selector on/off and docs-only changes. No live jobs launched.

AI assistance disclosure

OpenAI Codex (GPT-6) drafted the change.

Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
@taneem-ibrahim
taneem-ibrahim marked this pull request as ready for review November 12, 2026 19:33
@taneem-ibrahim taneem-ibrahim changed the title Add opt-in CI jobs without replacing normal selection [CI] Add opt-in CI jobs without replacing normal selection Nov 12, 2026
@taneem-ibrahim taneem-ibrahim changed the title [CI] Add opt-in CI jobs without replacing normal selection Add opt-in CI jobs without replacing normal selection Nov 13, 2026
khluu and others added 2 commits November 13, 2026 09:21
legacy_keys is what today's rules run, the baseline the selector is
scored against. _step_should_run now returns True for included keys, so
every /ci run --include build counted the requested jobs as today's
rules. Compute the baseline without them and record them separately as
included_keys.

Tests cover: included jobs run beside the selector's choice, stay out
of legacy_keys, an unknown key keeps normal CI, the docs-only skip is
bypassed, include + only is rejected, NOAUTO still defers, and the
parse error names the include variable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: khluu <khluu000@gmail.com>
Resolve the docs-only conflict by using main's _skips_ci_as_docs_only,
which now also keeps the build when /ci run --include asks for jobs, and
read the job steps once, before the include keys are validated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: khluu <khluu000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants