Skip to content

refactor(create): keep pure-Python operator sources in operators/<op>/ - #296

Merged
wyli merged 4 commits into
mainfrom
devin/1794001995-pure-python-operators-dir
Nov 8, 2026
Merged

wyli merged 4 commits into
mainfrom
devin/1794001995-pure-python-operators-dir

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Moves the pure-Python Module (holoscan create --language python, #284) operator source next to its metadata. Each operator is gated on OP_<name> through add_holohub_operator(), as C++ operators are. The layout now matches the C++ and 4.x templates:

before                                         after
python/holoscan_<slug>/operators/__init__.py   operators/CMakeLists.txt        # add_holohub_operator(<op>)
python/holoscan_<slug>/operators/<op>.py       operators/__init__.py           # exports staged operators
operators/<op>/metadata.json                   operators/<op>/CMakeLists.txt   # holoscan_<slug>_add_python_operator()
                                               operators/<op>/__init__.py
                                               operators/<op>/<op>.py
                                               operators/<op>/metadata.json

python/holoscan_<slug>/ keeps only __init__.py and py.typed. Imports don't change: from holoscan_<slug>.operators import <Op> and import holoscan.<slug> both still work.

  • Build tree: holoscan_<slug>_add_python_operator() (python/CMakeLists.txt) stages the calling operators/<op>/ sources as holoscan_<slug>.operators.<op>. Only operators whose OP_<op> is ON are staged. Each configure clears earlier staged operators.
  • Wheel: it installs the CMake component, the same way the cpp python wheel does:
    wheel.packages = []
    wheel.platlib = false
    install.components = ["holoscan-<slug>-python_libs"]
    cmake.define.BUILD_ALL = "OFF"
    cmake.define.OP_<op> = "ON"
    cmake.define.<SLUG>_BUILD_TESTING = "OFF"
    
    [[tool.scikit-build.overrides]]
    if.state = "editable"
    build-dir = "build/editable"
    The root CMakeLists.txt no longer returns early for SKBUILD. Dist-info is written only for non-wheel builds, because scikit-build-core supplies its own.
  • Editable installs stay live and gated: the generated docs use
    pip install --no-build-isolation -Ceditable.rebuild=true -e . (after pip install scikit-build-core~=1.0.3 cmake ninja). The package re-stages enabled operators on import, at a cost of about 0.2 s per import.
    • editable.rebuild is passed on the command line, not set in pyproject.toml. With build isolation, the rebuild can't find the isolated ninja and the import fails. Passing the flag keeps a plain pip install -e . working, as a copy.
  • operators/__init__.py exports every staged operator package at runtime using pkgutil, so disabling an operator needs no edit there. It also has one if TYPE_CHECKING: import per operator (from .<op> import <Op> as <Op>), so mypy, pyright and IDEs resolve the exports.
  • create and the post-gen hook point at operators/ and operators/<op>/<op>.py.
  • cpp and cpp python output is unchanged apart from the CLI version pin.

Verification

  • Unit tests: 768 passed, 1 skipped. A new check configures with -DOP_my_mod_op=OFF and asserts nothing is staged. Pre-commit passes on changed files.
  • Generated cpp and cpp python Modules with main and with this branch, then diffed: identical apart from the holoscan-cli== pin.
  • --language python Module in holoscan:v5.0.0-cuda13 (no GPU, stub libcuda), using a CLI wheel built from this branch:
Step Result
build and run --local Run printed received 10 samples; staged operators/my_module_op/
test --local CTest passed
install --dev --local from holoscan.my_module.operators import MyModuleOp worked
package --pkg-generator WHEEL py3-none-any wheel with operators/my_module_op/{__init__,my_module_op}.py and no metadata.json
pip wheel . -Ccmake.define.OP_my_module_op=OFF Wheel contained only operators/__init__.py
Fresh venv: install the wheel, pip check, pytest tests/python 3 tests passed
mypy / pyright on from holoscan_my_module.operators import MyModuleOp No errors; resolves to MyModuleOp
Documented editable install, then edit my_module_op.py Edit visible on next import; 3 tests passed
Same, with -Ccmake.define.OP_my_module_op=OFF Operator not exported
Plain pip install -e . 3 tests passed
ruff check / ruff format --check on the generated Module Clean

wyli also ran a GPU end-to-end check at 8bfa3d1 for both cpp python and --language python Modules; see his review comment.

Link to Devin session: https://nvidia-cloud.devinenterprise.com/sessions/e2d0e9d75b0b42f295b43491ae274a9d
Open in Devin Desktop: https://nvidia-cloud.devinenterprise.com/desktop/session/e2d0e9d75b0b42f295b43491ae274a9d?variant=devin

Signed-off-by: Tom Birdsong <tbirdsong@nvidia.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@copy-pr-bot

copy-pr-bot Bot commented Nov 6, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

…ith OP_<name>

Signed-off-by: Tom Birdsong <tbirdsong@nvidia.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Tom Birdsong <tbirdsong@nvidia.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@wyli wyli left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked at 8bfa3d1. Unit tests pass, and an end-to-end run on a GPU host with holoscan:v5.0.0-cuda13 passed for both a cpp python and a --language python Module: create, build, run, CTest, install --dev, the WHEEL package, and the wheel consumer. Locally, the staging, the install component, the wheel contents, OP_<op>=OFF, and restaging after adding, renaming, or deleting an operator all behave as described.

One fix is needed before merge; see the inline comment. Type checkers no longer see the operators.

Signed-off-by: Tom Birdsong <tbirdsong@nvidia.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@wyli
wyli merged commit 61581b4 into main Nov 8, 2026
27 checks passed
@wyli
wyli deleted the devin/1794001995-pure-python-operators-dir branch November 8, 2026 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants