How does development mode handle .pyd files and exclusion config?

[TSoli]

Hi, I am currently using hatch in a project with hatch-cython to compile parts of it. I have been able to set up the hatch-cython hook such that it compiles the files I desire and excludes some (using its exclude option) like

[tool.hatch.build.targets.wheel.hooks.cython.options.files]
exclude = ["./srs/package/my_file.py"]

Then I can choose which are excluded when I build the wheel

[tool.hatch.build.targets.wheel]
ignore-vcs = true
exclude = ["src/**/*.py", "!src/**/my_file.py", "src/**/*.c"]

Above I exclude the .py files because I will instead have compiled .pyd files. I then tell it not to exclude my_file.py because I do not have a compiled version of it from cython.

This seems to work correctly and if I hatch build and then pip install --target my_pkg my_pkg.whl then I can see it builds the wheel as I specify.

On the other hand, when I run in a test environment and it installs the package in development mode, only the compiled pyd files are included and my_file.py is not in the installed directory C:\Users\<user>\AppData\Local\hatch\env\virtual\my_pkg\<id>\test\Lib\site-packages\my_pkg.

Some related config for my test env

# Test env
[tool.hatch.envs.test]

dependencies = [
    "pytest",
    "pytest-azurepipelines",
    "coverage[toml]",
    "ruff",
    "pytest-cov",
]

[tool.hatch.envs.test.scripts]
lint = "ruff check"
test-cov = "coverage run -m pytest {args:tests}"
cov-report = ["- coverage combine", "coverage xml"]
cov = ["test-cov", "cov-report"]

# Coverage
[tool.coverage.run]
source_pkgs = ["my_pkg", "tests"]
branch = true
parallel = true
omit = ["src/my_pkg/__about__.py"]

[tool.coverage.paths]
my_pkg = [
    "src/my_pkg",
    "*/my_pkg/src/my_pkg",
]
tests = ["tests", "*/my_pkg/tests"]

[tool.coverage.report]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

Note that if I add dev-mode = false in tool.hatch.envs.test then it builds correctly (I suppose because it is building from the wheel instead). How, then, is the package built in development mode and how come it is not following the exclude rules I set for the wheel? I notice that it still seems to trigger the cython hook I setup for wheel and generates the files correctly (i.e it is compiling everything except my_file.py).

I might update with a minimum reproducible example later if it helps but if anyone even has some info on exactly how dev builds are set up that would be useful info.

[tayfuryldz]

The key difference is that Hatch's dev mode is an editable wheel, not a normal wheel copied into site-packages. So checking whether my_file.py physically exists under ...Lib/site-packages/my_pkg is not a reliable test for an editable install.

Current Hatch/Hatchling documents the wheel target's editable version as a wheel that ships .pth files or import hooks pointing Python at the working tree. For dev-mode installs, the wheel target determines which source directories to put on Python's search path from the selected files. That is deliberately different from the normal wheel file-selection/copy operation.

First check the import rather than the site-packages directory:

hatch run test:python -c "import my_pkg.my_file as m; print(m.__file__)"

For an editable install, seeing that resolve into your checkout (for example ...\src\my_pkg\my_file.py) is expected.

There is also an explicit escape hatch when automatic dev-mode path detection is confused by unusual build-hook/file-selection setups such as “exclude Python sources from the wheel because Cython emits .pyd files”:

[tool.hatch.build.targets.wheel]
dev-mode-dirs = ["src"]

Current Hatch docs describe dev-mode-dirs specifically as overriding which directories are added to Python's search path for editable/dev-mode installs.

So I would separate the two desired artifacts:

• release wheel: your exclude/negation rules decide which source/compiled files are physically shipped;
• Hatch test environment in dev mode: source files normally remain in the checkout and become importable through the editable path/import hook.

That also explains why dev-mode = false appears to “fix” the physical layout: you have stopped asking for an editable install, so the environment receives a normal built artifact instead.