[nvrithik]

Changelog title

More reliable vLLM setup and model loading across supported devices

Changelog body

• Install PAIR-managed vLLM outside the space-containing app-data directory to avoid GPU-kernel compilation failures.
• Size vLLM memory allocations using reliable device readings, including verified shared-memory systems.

Bumps

• services: minor
• nvpair-cluster-manager: none
• nvpair-engine-manager: patch
• nvpair-errors: none
• nvpair-job-scheduler: none
• nvpair-manual-nodes: none
• nvpair-node-info: none
• nvpair-node-scanner: none
• nvpair-node-settings: none
• nvpair-proxy: none
• nvpair-tui: none
• nvpair-ui-broker: none
• nvpair-workload-manager: none

Description

Fix vLLM model-startup failures caused by paths containing spaces and unreliable device-memory readings.

Install PAIR-managed vLLM under ~/.nvpair/engine-bin/vllm instead of the space-containing app-data directory, and put its virtual-environment tools on PATH. Reset and uninstall remove this managed environment while preserving downloaded Hugging Face models.

The location change works around upstream compiler-command quoting problems:

• FlashInfer #3515 documents a related host-compiler path failure.
• DeepJIT #22 proposes a fix for unquoted compiler arguments and working directories in the runtime used by DeepGEMM.

The workaround avoids affected installation paths while upstream fixes work their way into PAIR’s vLLM dependency.

For memory sizing, use valid NVIDIA telemetry when available and otherwise query the installed runtime. Use system RAM only after confirming shared memory, reserve an additional 3 GiB for startup on those devices, and leave the memory limit to vLLM when reliable readings are unavailable.

Validation

• Focused engine-manager tests:
  go test ./... -run 'Test(AutoContext|ApplyAutoLaunch|PlanToolCalling|ParsesToolCalls|DetectDeviceMemory|ParseDeviceMemoryProbe|GPUMemoryFromQuery|PlanHFContext_|PlanMetalKVCache|ValidateMetalModel|ProcessEnvironment|VenvBinDir|StartManagedProc_FindsTools|Validate_.*InstallDirectory|ApplyPlatformDefaults_|UninstallManaged_|BundledVLLMManifest|BundledModelStores_)' -count=1
• Cleanup inventory: npx --no-install vitest run tests/modular/app-data-wipe.test.ts
  (12 tests passed across the configured unit and e2e projects).
• Packaging/static checks: bash -n on the three shell cleanup scripts,
  scripts/wipe-app-data.sh --dry-run, Python AST parsing for the embedded probe,
  Prettier check, git diff --check, and node scripts/spdx-headers.mjs.
• Cross-builds: GOOS=darwin GOARCH=arm64 go build . and
  GOOS=windows GOARCH=amd64 go build . from services/nvpair-engine-manager.
• DGX Spark, real PAIR install: fresh vLLM install created the executable and
  ownership marker under ~/.nvpair/engine-bin/vllm. With kernels enabled,
  nvidia/Llama-3.1-8B-Instruct-NVFP4 and
  nvidia/Llama-3.1-8B-Instruct-FP8 both reached public/internal health 200
  and answered inference requests. Logs contained no nvcc fatal,
  ninja: error, readiness exit, NotImplementedError, or OOM.
• Apple M4, real vLLM-Metal install: reproduced the reported FP8 KV-cache error;
  forcing a BF16 KV cache passed that check but then proved the ModelOpt FP8
  weights themselves are unsupported. The final manager rejected the checkpoint
  in about 0.007 s, with no vLLM child or listening port. The new memory probe
  also reported the machine's 48 GiB unified memory and current free memory.

The full repository suite was not run; validation was intentionally focused on
the three fixes and their packaging paths.

Checklist

• I have read the Contributing Guidelines.
• Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
• New or existing tests cover the change.
• Relevant documentation is updated.
• I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
• I recorded the validation commands and results above.
• I declared version bumps in the release-intent block above. services/versions.json is written by automation — do not edit it by hand.

[nvrithik]

Clean runtime validation

Tested commit 75be10ad25652e23e45d9c439547bc5aa3bf81e0 from fresh clones. Before testing, I removed the previous PAIR-managed vLLM installations, engine desired state, and generated vLLM/FlashInfer code caches on both hosts. Hugging Face model caches were intentionally preserved.

DGX Spark / Linux arm64

• Fresh managed install completed at ~/.nvpair/engine-bin/vllm in 233.7 seconds; the legacy space-containing path was absent.
• The installed vLLM platform probe reported 130,662,936,576 bytes total, 122,962,579,456 bytes free, and shared/unified memory.
• nvidia/Llama-3.1-8B-Instruct-NVFP4 became healthy in 248.1 seconds and returned HTTP 200 from chat inference.
• nvidia/Llama-3.1-8B-Instruct-FP8 became healthy in 274.5 seconds and returned HTTP 200 with PAIR FP8 READY.
• Both launches used the space-free managed Python/vLLM path, put its venv/bin first on PATH, and received automatic GPU-memory utilization values.
• No nvcc fatal, Ninja error, readiness exit, NotImplementedError, or out-of-memory pattern appeared in the captured logs.

Apple M4 / macOS arm64

• Fresh managed install completed at ~/.nvpair/engine-bin/vllm in 47.4 seconds; the legacy Application Support engine path was absent.
• The installed vLLM Metal probe reported 51,539,607,552 bytes total, 36,596,203,520 bytes free, and shared/unified memory.
• Loading nvidia/Llama-3.1-8B-Instruct-FP8 was rejected by PAIR's preflight in 0.019 seconds with the expected ModelOpt FP8 / vLLM Metal compatibility guidance.
• No vLLM child process was spawned and neither the public nor internal health port was left listening.

Both fresh source checkouts remained clean at the tested SHA, both managed-install markers were present, and no PAIR/vLLM test process remained afterward.