Skip to content

feat(python): type stubs, InvalidInput for unrepresentable values, MIT licence - #575

Merged
yifanxuaaa merged 6 commits into
layerfs_v2_0_0from
issue30/python
Nov 5, 2026
Merged

yifanxuaaa merged 6 commits into
layerfs_v2_0_0from
issue30/python

Conversation

@Mola-maker

Copy link
Copy Markdown
Member

Summary

The Python package had no type information, turned some unrepresentable values into raw UnicodeEncodeError / TypeError, and carried no licence file. This refuses such values as LayerFSError("InvalidInput") before any native call, ships py.typed and .pyi stubs generated from the call table, and ships the MIT licence. Python integers were already exact and are unchanged.

Refs Ephemeral-AI-Lab/layerfs-project#35 (Python SDK), parent Ephemeral-AI-Lab/layerfs-project#30. Findings and decisions: https://github.com/Ephemeral-AI-Lab/layerfs-project/issues/31#issuecomment-6657201351

Changes (5 commits)

  • fix(python): NaN, Infinity, lone surrogates (also in a home or scope), non-text map keys and types JSON has no form for raise InvalidInput before the call. An invalid base64 one-key map reads back as a map instead of raising.
  • feat(python): py.typed; .pyi for the sync and async clients, generated by api/python/tools/gen_stubs.py from the table the native module exposes; _values.pyi and _errors.pyi by hand. Results are typed loosely (Any) because the call table has no result schema; the Rust call table is untouched. The async client declares no home/scope, as it sets none.
  • chore(python): license = "MIT" and license-files; LICENSE copied into the package (maturin does not allow a .. path).
  • test(python): values, bytes and integers, a mountless sync and async flow, stub drift, every attribute a client stub declares exists on the client, strict mypy on a correct and a misusing sample.
  • docs(python): README and front-ends/python.md.

Verification (WSL Ubuntu, Python 3.14, maturin 1.16, PyO3 0.30.1)

  • Feasibility: maturin develop --locked --offline built the module in 1m29s with no ABI3 forward-compat flag.
  • Baseline suite: 9 pass, 1 skipped. At the final commit 8547763bf: python -m unittest discover -s test 39 tests OK, 1 skipped (needs LAYERFS_DAEMON, not run); gen_stubs.py --check exit 0.
  • mypy --strict and pyright on ok.py: 0 errors; bad.py under mypy reports 14 errors, exactly the lines marked # E. The built wheel installed in a clean venv passes mypy without MYPYPATH. pip show reports License-Expression: MIT; the wheel has dist-info/licenses/LICENSE and the stubs and py.typed; the sdist has LICENSE.
  • Red/green for the async stub defect: with the old stub restored the new test fails with aio.pyi: AsyncClient.home is not on the client.
  • cargo fmt --all -- --check exit 0, check_product_boundary.py PASS. No Rust changed, so clippy and Rust tests were not run. Production LOC: 0 (the counter does not count Python); hand-written .py net about +40 lines.
  • An independent review by a separate model had three minor points: the async stub defect was real and is fixed; license = "MIT" (PEP 639, maturin>=1.9.6 is required) was declined; a hard-coded repo-root path in the licence test is now skipped outside the checkout.
  • /tmp on WSL is tmpfs and SQLite locking is unsupported there: set TMPDIR to ext4 (documented).

Not done / risks

  • No call echoes an i64 boundary value (line and page options are capped at 2^53-1), so end-to-end exact echo cannot be shown; only "reaches the call" and decode exactness are tested.
  • file.read on binary content errors (baseline behaviour). {"base64": "..."} as content is read as bytes, not a map (documented).
  • Executable Python examples were not written. Event-loop and cancellation behaviour is untouched (owned by control-plane [v0.1.3][Stage 1][01] Build and baseline payload creation and random reads #23).
  • A local dev tree's __pycache__ ended up in a wheel; a clean checkout does not have it.

🤖 Generated with Claude Code

root and others added 5 commits November 5, 2026 08:14
NaN, Infinity, lone surrogates (also in a home or scope), non-text map keys and types JSON has no form for now raise LayerFSError InvalidInput before any native call, instead of a raw UnicodeEncodeError or TypeError. An invalid base64 one-key map is read back as a map.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tools/gen_stubs.py writes the sync and async stubs from the table the native module exposes; the async client declares no home or scope, as it sets none. Results are typed loosely because the table has no result schema.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
license is the SPDX string MIT and the repository's LICENSE is part of the sdist and wheel.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Cover the refused values, bytes and integers, a mountless sync and async flow on a temporary home, stub drift, every attribute a client stub declares, and strict mypy on a correct and a misusing sample.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Describe what is refused before a call, the stubs and how to regenerate them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
yifanxuaaa added a commit that referenced this pull request Nov 5, 2026
Limit the mutable sink list to the Linux-only block.

Verified on macOS: all 21 CLI tests pass; Clippy all targets with -D warnings passes. No tests deleted.

Production LOC: 137229 -> 137229 (delta +0)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
An omitted callback stays optional; an explicitly supplied None is no longer accepted by the sync or async stubs. Add mypy regression cases and regenerate both stubs. Extend the production counter to cover Python source and handwritten stubs, excluding comments, docstrings and generated stubs.

Verified: Python 39 tests, 38 pass and one daemon-dependent skip, including strict mypy positive/negative samples; stub drift check; 169 tooling tests; pinned Ruff. No tests deleted.

Production LOC: 137276 -> 137276 (delta +0)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
yifanxuaaa added a commit that referenced this pull request Nov 5, 2026
Accept Uint8Array, including Buffer, in Value. Cover binary file.apply in the type sample and real operation tests. Align the versioned manual and SDK skill with Node >=22.

Verified: Node 24 tests, 23 pass and one daemon-dependent skip; TypeScript 7.0.2 strict sample and generator drift check; runtime image Node 24.21.0. No tests deleted.

Production LOC: 137245 -> 137245 (delta +0)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
yifanxuaaa pushed a commit that referenced this pull request Nov 5, 2026
Apply the existing scope reference check to branch.fork from_commit and branch.publish commit. Add real-operation scope and pagination tests, and document the remaining scope boundaries.

Validation on the combined final tree, macOS ARM64/Rust 1.85.1: calls_scope 4, scope_references 5, list_pages 3, commit_get_pages 1, calls_table 15 passed. Warning-denying Clippy across API/control/CLI/Node/Python all targets, Rust formatting, product boundary guard and 169 tooling tests passed.

Tests used Disposable Stores; Durable execution was NOT_RUN, disabled by owner. No tests deleted. Daemon-dependent SDK execution is not claimed.

Production LOC: 137217 -> 137223 (delta +6)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
yifanxuaaa pushed a commit that referenced this pull request Nov 5, 2026
Read argv as OS strings and refuse non-UTF-8 words with exit 2. Write help, version and diagnostics through fallible output helpers. Treat doctor --help as help. Keep existing exit-code policy. Fix the failing-sink fixture so macOS compiles warning-free.

Validation on the combined final tree: all 21 CLI tests passed on macOS after a separate no-run build; warning-denying Clippy all targets and Rust formatting passed.

Tests used Disposable Stores; Durable execution was NOT_RUN, disabled by owner. No tests deleted. Daemon-dependent SDK execution is not claimed.

Production LOC: 137223 -> 137235 (delta +12)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
@yifanxuaaa
yifanxuaaa merged commit 8c31f1a into layerfs_v2_0_0 Nov 5, 2026
3 of 5 checks passed
yifanxuaaa pushed a commit that referenced this pull request Nov 5, 2026
Preserve exact bigint values, reject unrepresentable inputs, load only the packaged addon or LAYERFS_NATIVE, and ship MIT licensing plus generated declarations. Accept Uint8Array inside JSON arguments such as file.apply. Set Node >=22 consistently in package metadata, the versioned manual and SDK skill.

Validation on the combined final tree: Node 24 tests completed, 23 passed and one daemon-dependent exec test skipped; TypeScript 7.0.2 strict sample and declaration drift check passed. The registered runtime image runs Node 24.21.0. Native CLI/Node/Python artifacts were freshly built with locked offline dependencies in the registered private worktree.

Tests used Disposable Stores; Durable execution was NOT_RUN, disabled by owner. No tests deleted. Daemon-dependent SDK execution is not claimed.

Production LOC: 137294 -> 137322 (delta +28)

Scope: first-party core, engine, connect, API Rust/SQL/JS and shipped Python
implementation plus handwritten Python stubs. Generated stubs, tests, tools,
examples, docs and third-party code excluded. Python comments and documentation
strings excluded. Counter blob 31b3cc2 (tools/production_loc.py
extended in #575), identical for both exact exported snapshots.
Command: python3 tools/production_loc.py --root <git-archive-tree> --json.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
@yifanxuaaa
yifanxuaaa deleted the issue30/python branch November 18, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants