Skip to content

[nvidia_plugin] Migrate CI to isolated per-module workflow - #1126

Closed
allnes wants to merge 1 commit into
openvinotoolkit:masterfrom
allnes:ci/module-nvidia-plugin
Closed

allnes wants to merge 1 commit into
openvinotoolkit:masterfrom
allnes:ci/module-nvidia-plugin

Conversation

@allnes

@allnes allnes commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Introduces a small module-oriented CI entry point for openvino_contrib, with nvidia_plugin as the first independent module workflow.

Architecture

  • overall_status.yml is the repository orchestrator. It runs the existing OpenVINO Smart CI implementation to select affected modules, calls each module workflow as an equal peer, and aggregates their results in ci/gha_overall_status.
  • .github/components.yml contains only Smart CI dependency relationships. Unknown modules are validated conservatively until their own workflow is registered.
  • module_<name>.yml owns the complete readable flow for that module: images, runners, containers, build commands, artifacts, hardware and tests. There is no generic lifecycle with a growing input list and no custom CI DSL.
  • Adding a module requires its category/dependency mapping, one module workflow, and one explicit call from the orchestrator. Existing module workflows do not gain module-specific switches.

NVIDIA migration

  • Moves NVIDIA build and GPU validation out of the legacy Linux job.
  • Builds against the prebuilt OpenVINO package and uses a branch-scoped sccache.
  • Transfers only the test binaries and required runtime libraries through the official upload/download artifact actions.
  • Runs NVIDIA unit tests and smoke functional tests on every selected pull request. Nightly validation adds the full functional suite and Compute Sanitizer.
  • Keeps build and test commands directly visible in module_nvidia_plugin.yml; the only Python helper validates the module-generated plugin registry and collects the minimal runtime for transfer.
  • Removes obsolete test_cuda.yml, sanitizer_cuda.yml, and history_cuda.yml workflows.
  • Includes CUDA 11.8 compatibility fixes found by the real GPU run and preserves sanitizer arguments as Bash arrays.

Safety

  • Workflows have read-only repository, pull-request and artifact permissions and do not receive repository secrets.
  • The GPU job uses a pinned, read-only, capability-dropped container with no-new-privileges, a PID limit, bounded temporary storage, and only the NVIDIA devices exposed.
  • Checkout credentials are not persisted, the OpenVINO artifact mount is read-only, and external actions and runtime container images are pinned.
  • Smart CI dependencies are installed with runner-managed Python; no deprecated Node 20 compatibility override is used.

@allnes
allnes requested a review from a team as a code owner August 4, 2026 20:14
@github-actions github-actions Bot added the category: CI OpenVINO public CI label Aug 4, 2026
@allnes
allnes requested review from a team as code owners August 4, 2026 21:22
@github-actions github-actions Bot added the category: NVIDIA plugin OpenVINO NVIDIA plugin label Aug 6, 2026
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch 3 times, most recently from 6cf455a to a2fe261 Compare August 10, 2026 13:03
@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Aug 10, 2026
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch from e52499b to 144de2f Compare August 10, 2026 15:54
@allnes allnes added category: CI OpenVINO public CI and removed category: CI OpenVINO public CI labels Aug 11, 2026
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch 14 times, most recently from a582b10 to 2721563 Compare August 19, 2026 17:41
@akladiev
akladiev requested a review from akashchi August 21, 2026 10:40
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch from 2721563 to 0a1e855 Compare August 21, 2026 12:09

@akashchi akashchi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have serious reservations about the overall direction. My core concern is that this represents a global over-complication of the CI that will further discourage developers from adding tests themselves, since it introduces a level of obscurity for the CI setup.

We deliberately moved away from this kind of complexity when we migrated the tests out of Jenkins into openvino GHA CI.

My main takeaways are:

  1. Re-implementing native GHA stuff handling in Python scripts

E.g., the PR moves artifact handling into custom Python scripts (e.g. the pull & build logic in module_openvino_plugin.yml) and container orchestration into .github/scripts/common/container_job.py. Why? GitHub Actions already provides this. I trust actions/download-artifact far more than a non-native (from a GitHub Actions perspective) approach to downloading artifacts. We have already implemented that in the Smart CI action. GitHub admittedly doesn't handle artifacts (download/upload) perfectly on its own, but the answer to that shouldn't be to route everything through hand-written Python.

  1. The README.md claims "native"

The README states at the very top:

The CI uses native reusable workflows instead of a configuration DSL

...and is then followed by a large amount of custom Python code that is arguably closer to a DSL than what we have today. We already have native .yml workflows.

  1. Re-inventing existing GitHub Actions

Several hand-written utilities re-implement functionality that already exists as off-the-shelf GitHub Actions, e.g., detect_git_changes. We should prefer maintained, community actions over bespoke scripts we now have to own and maintain ourselves.

  1. Magic strings

There are a lot of magic strings scattered across the workflows, e.g., contrib-ci-build. These are fragile, undocumented coupling points that are easy to break and hard to trace.

  1. Scalability problem

module_openvino_plugin.yml, which acts as the entry point, already has 18 inputs while serving only a single plugin. For comparison, our job_build_linux.yml in the openvino repository has 20 inputs but covers the whole Linux build. An 18-input adapter per module does not scale well and undermines the "short module-owned adapter" description in the PR summary.

  1. Further adoption

If this pattern lands fully in openvino_contrib, the implicit next step is porting it to other repositories.

  1. Debugging

A key benefit of native .yml workflows is that a contributor can read the workflow and understand exactly what runs w/o delving into custom scripts. Of course, there are actions and reusable workflows, but the general flow, imo, should be defined in a .yml.

@mryzhov, @akladiev, please advise.

@allnes
allnes force-pushed the ci/module-nvidia-plugin branch from 0a1e855 to fcae24a Compare September 4, 2026 16:43
@github-actions github-actions Bot removed the dependencies Pull requests that update a dependency file label Sep 4, 2026
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch 13 times, most recently from 1a3dd2c to 6cae740 Compare September 4, 2026 19:14
@allnes
allnes force-pushed the ci/module-nvidia-plugin branch from 6cae740 to 8ca4327 Compare September 4, 2026 19:36
@allnes
allnes requested a review from akashchi September 6, 2026 21:06
@allnes

allnes commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

@akashchi please review it again

persist-credentials: false

- name: Get prebuilt OpenVINO
uses: ./openvino_actions/.github/actions/restore_artifacts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use openvino_provider?

"${NVIDIA_ARTIFACT}/bin/cuda-sanitizer.sh"

container=(
docker run --rm

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate - why is it done like that? There's an option to schedule the entire GHA job within a container via "container" parameter, there should also be a flexibility for parameters setting.

build:
name: Build
needs: images
runs-on: aks-linux-16-cores-32gb

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably we could consider smaller and cheaper VM, if the build is not demanding for resources. @ababushk, what do you think?

Comment on lines +126 to +135
- name: Restore sccache
id: sccache
continue-on-error: true
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.SCCACHE_DIR }}
key: nvidia-ubuntu22-release-${{ github.ref_name }}-${{ github.sha }}
restore-keys: |
nvidia-ubuntu22-release-${{ github.ref_name }}-
nvidia-ubuntu22-release-master-

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had any activity for the last 2 weeks. It will be closed in 7 days if no further activity occurs. Please push new commits or leave a comment to keep it open. Thank you for your contributions!

@github-actions github-actions Bot added the stale No activity for 2 weeks; will be closed in 7 days label Sep 29, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

This pull request has been automatically closed because it has been stale for 7 days with no activity. Feel free to reopen it if you would like to continue the work. Thank you for your contributions!

@github-actions github-actions Bot closed this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: CI OpenVINO public CI category: NVIDIA plugin OpenVINO NVIDIA plugin stale No activity for 2 weeks; will be closed in 7 days

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants