Skip to content

feat(agent): cuinterpose shim symbol resolution and forwarding - #215

Closed
galletas1712 wants to merge 1 commit into
feat/cuinterpose-agentfrom
feat/cuinterpose-shim-forwarding
Closed

galletas1712 wants to merge 1 commit into
feat/cuinterpose-agentfrom
feat/cuinterpose-shim-forwarding

Conversation

@galletas1712

@galletas1712 galletas1712 commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Layer 5 of the eleven-PR cuinterpose stack #293. Closed PR #214 is intentionally not in the active stack.

This PR replaces the inert shim with transparent CUDA forwarding and defines the POSIX ticket primitives later tracking layers use. No allocation or multicast state is tracked at this layer.

Cuinterpose intercepts all four ways an application can reach a driver entry point: direct ELF binding, dlsym, the cuGetProcAddress* family, and the CUDA runtime's cudaGetDriverEntryPoint* family. Resolver wrappers call the real resolver first and substitute only successful CUDA results, so the shim never invents APIs unsupported by the installed driver. CUDA 13.1 requests select the device-explicit multicast bind ABI.

The intercepted surface covers CUDA VMM create/release/retain/map/unmap/access/export/import/property APIs and the corresponding multicast operations. Every wrapper checks symbol availability and forwards arguments/results unchanged in this PR.

posix.c defines the sealed 256-byte memfd ticket used in later layers. A ticket names the original creator participant and endpoint, resource kind, and random 128-bit allocation ID. The allocation ID is the opaque resource identity and export-cache key. An importer sends an EXPORT request carrying the participant, resource kind, and allocation identity needed for lookup. The creator rejects unknown IDs and returns a fresh real CUDA descriptor over SCM_RIGHTS for a known cached resource. Both peer traffic and coordinator traffic terminate on the same owner-only per-process Unix socket once #217 starts it. The peer request path uses scoped descriptor ownership and direct error returns; it has no goto cleanup path.

Tests cover direct and resolver forwarding, ABI selection, ticket validation, descriptor passing, malformed ancillary data, unavailable symbols, and the rule that resolver failure is not substituted.

Stack boundary

Based on #223. #216 implements coordinator orchestration and global validation. #217 turns the forwarding wrappers into active POSIX-FD tracking and starts the per-process listener.

Validation

On the final stack, make test passes in all Go modules. The pinned CUDA 13.1 builder compiles the production shim and coordinator with -Werror; the forwarding suite passes 12 tests, and the protocol suite passes 9 tests, with ASan/UBSan enabled. The final published stack head 0eae9c4f9b66093332f00a332afaf56048e1e64a passed the full make check gate, the CUDA 13.1 production build, and all 73 sanitizer-backed native cuinterpose tests. Its physical-GPU suite passed all 3 tests with no skips on two DRA-assigned NVIDIA B200 GPUs. The unicast test kept an explicit allocation-ID ticket-backed peer mapping per worker live across capture and restore; the multicast test required a nonzero multicast VA and shim-logical handle, exercised BindAddr, and passed its collective and captured-graph replay. Detailed hardware evidence and measurements are in #220.

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@copy-pr-bot

copy-pr-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from 78372a1 to c88e8fc Compare September 11, 2026 06:10
@galletas1712
galletas1712 removed this pull request from stack #221 September 11, 2026 06:11
@galletas1712
galletas1712 added this pull request to stack #293 September 11, 2026 06:11
@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from c88e8fc to c6f98ce Compare September 11, 2026 06:16
@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from c6f98ce to 4dc1da4 Compare September 11, 2026 06:21
@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from 4dc1da4 to 10ff15b Compare September 11, 2026 06:41
@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from 10ff15b to 54fc9cf Compare September 11, 2026 08:31
@galletas1712
galletas1712 force-pushed the feat/cuinterpose-shim-forwarding branch from 54fc9cf to 622e6c5 Compare September 11, 2026 09:41
Replace the inert placeholder with a shim that intercepts CUDA driver entry
points reached through direct linking, dlsym, cuGetProcAddress variants, and
cudaGetDriverEntryPoint variants. Ask the real resolver first and substitute
only APIs supported by the installed driver. Build against CUDA 13.1 headers
for the device-explicit multicast bind ABI without linking libcuda or libcudart.

Define the sealed 256-byte POSIX ticket used by later tracking layers. It names
the creator participant, private control endpoint, resource kind, and random
128-bit allocation ID. The allocation ID is the opaque bearer capability and
future export-cache key; there is no separate authorization field. Peer export
requests return one validated SCM_RIGHTS descriptor and use scoped cleanup on
all error paths.

Keep generic C infrastructure under util/, suppress SIGPIPE on socket writes,
close descriptors from malformed messages, and test every forwarding route and
ticket invariant with fake CUDA libraries under sanitizers.

Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
@galletas1712

Copy link
Copy Markdown
Contributor Author

Superseded by the replacement 13-PR C-frontend/Rust-backend cuinterpose draft stack: https://github.com/ai-dynamo/snapshot/stack/339 (#326#338), tracking approved proposal #295. The new stack preserves one reviewable stage per PR, with tests collected in #338 and the design in #337. PageBroker transfer changes remain a separate dependency. Closing this older C implementation; retaining its branch and discussion for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants