Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion services/nvpair-node-info/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Field notes:
- `power_watts` is what a device is drawing in whole watts, published only where the hardware meters itself: an NVIDIA GPU (`nvidia-smi`'s `power.draw`, Linux and Windows), an AMD discrete GPU (the amdgpu hwmon's `PPT` input), and on `cpu` the package power — derived from the processor's energy counter, or on a Linux AMD APU host from that APU's `PPT` input, which meters the whole socket. Every other row has no meter and carries no such field — see **Power draw** below for the full table and for why a Linux host normally reports no `cpu.power_watts`.
- All dynamic fields and the `cpu` / `memory` objects use `omitempty`: a value the service couldn't read is dropped from the JSON entirely rather than reported as a misleading literal zero. A genuinely idle CPU renders the same as "unknown" — that ambiguity is intentional and benign.
- `utilization_unavailable` is `true` on a device row that has **no busy counter** this service can read, and absent everywhere else. It exists because `utilization_percent` is `omitempty`: a measured idle `0` is absent on the wire too, so the absence alone cannot tell "idle" from "cannot tell", and a client that renders both as "0 %" claims an idle device nobody measured. Set on a Hailo module for which neither the inference process's activity file nor the `nvpair-sensors` I/O-activity estimate gives a current figure (HailoRT has no busy counter on Windows; see [Utilization from the activity file](#utilization-from-the-activity-file) and [Utilization from the I/O-activity estimate](#utilization-from-the-io-activity-estimate)), on a Linux Intel GPU whose driver publishes no idle-residency counter and, until its second sample, on one that does (i915/xe keep their engine-busy counters in a root-gated PMU; the figure is derived from RC6 / GT idle residency instead, see [Linux Intel (i915 / xe sysfs)](#linux-intel-i915--xe-sysfs)), and on a Mali GPU or RKNPU row whose sampler has not read its counter — before the first read, or after three consecutive failed reads, rather than freezing the last value. A client must show such a row's usage as unknown; a row without the flag keeps its historical meaning, so an idle GPU on a node that predates the field still reads 0 %.
- `inference_ready` is `false` on a device no engine PAIR runs can use — an Arm Mali GPU, an RKNPU, a Google Coral Edge TPU, a Hailo module, a Linux Intel integrated GPU — and absent on every other row, which makes no claim either way. It travels per row rather than as a node-level id list because clients re-sort the rows and build their own ids. The desktop derives the node's inference-ready list from it, so a board whose only devices are a Mali GPU and an NPU shows its CPU and RAM instead of two idle GPU rings.
- `inference_ready` is `false` on a device no engine PAIR runs can use — an Arm Mali GPU, an RKNPU, a Google Coral Edge TPU, an Axera AX650N card, a Hailo module, a Linux Intel integrated GPU — and absent on every other row, which makes no claim either way. It travels per row rather than as a node-level id list because clients re-sort the rows and build their own ids. The desktop derives the node's inference-ready list from it, so a board whose only devices are a Mali GPU and an NPU shows its CPU and RAM instead of two idle GPU rings.
- `memory.total_bytes` and `memory.used_bytes` are on **one base**: the memory the operating system manages, read from the same source the used figure is — `/proc/meminfo` `MemTotal` on Linux (used = `MemTotal - MemAvailable`), `GlobalMemoryStatusEx` `TotalPhys` on Windows (used = `TotalPhys - AvailPhys`), gopsutil's Mach readers on macOS. The Linux Intel, Mali, RKNPU and NVIDIA UMA rows use this same total as their `vram_bytes` ceiling; a Windows integrated GPU does not (see `memory_pool` below), because this base excludes the firmware-reserved memory an APU's graphics carve-out lives in. Installed DIMM capacity is not used: on Windows it counts hardware-reserved memory that can never appear as used, and on Linux it is not readable unprivileged. The Linux total used to be the count of online memory blocks times the block size, which counts the blocks around the PCI hole and at the top of RAM in full: a 64 GiB desktop with 2 GiB blocks published 66 GiB against a `MemTotal` of 62.7 GiB, and RAM % read two points low.
- `vram_bytes` is reported through DXGI on Windows, `nvidia-smi` on Linux, and IORegistry on macOS. On a unified-memory NVIDIA GPU such as DGX Spark, Linux uses total physical system memory for `vram_bytes` and the independently sampled system-memory usage for `vram_used_bytes`. On Apple Silicon, `vram_bytes` is total physical unified memory and `vram_used_bytes` is the GPU driver's mapped allocation (`Alloc system memory`), not whole-system RAM usage or the momentarily active subset.
- `memory_pool` is absent on a device with memory of its own and `"unified"` on one whose `vram_bytes` is a pool it shares with the host. A client must not label a unified capacity "VRAM", and must not assume such a row reports usage — see **Unified memory rows** below. On Windows an Intel or AMD integrated adapter is classified by the same PCI id table the Linux inventory uses (`nvpair-shared/gpunames`), and an id the table does not list by DXCore's `IsIntegrated` adapter property; such a row's `vram_bytes` is DXGI's `DedicatedVideoMemory + SharedSystemMemory` and its `vram_used_bytes` is PDH's `Dedicated Usage + Shared Usage` — the Windows counterpart of amdgpu's VRAM + GTT on Linux. The dedicated half alone is only the firmware's stolen aperture (128 MB on a UHD 630) or an APU's carve-out, and the host memory total would lose that carve-out (it excludes firmware-reserved memory), so neither is used as the ceiling. Without a `Shared Usage` sample the row publishes no used figure rather than the dedicated half alone. A startup row the OS could not classify takes the integrated classification from a later re-detection; a later pass that cannot say does not undo it.
Expand All @@ -130,6 +130,7 @@ Several devices in this inventory have no memory of their own: an Intel or AMD i
| Intel integrated GPU | i915/xe expose no power attribute to an unprivileged reader | no |
| Arm Mali GPU, RKNPU | no meter in the driver | no |
| Google Coral Edge TPU | the gasket/apex driver keeps no power attribute | no |
| Axera AX650N card | `axcl-smi` reports `-- / --` for power | no |
| Hailo-8L | HailoRT's `measure-power` is unsupported on that module | no |
| Apple Silicon GPU | IOAccelerator publishes no power figure | no |

Expand Down Expand Up @@ -405,6 +406,30 @@ GOOS=linux GOARCH=arm64 go test -c -o rockchip_arm64.test # cross-compile
NVPAIR_LIVE_ROCKCHIP=1 ./rockchip_arm64.test -test.v # on the board
```

### Linux Axera (AX650N accelerator cards via `axcl-smi`)

An Axera AX650N-based card (for example the M.2 "AX8850"; PCI vendor `0x1f4b`, device `0x0650`, driven by the vendor's `axcl_host` modules) is listed as an accelerator row — `kind:"npu"`, `inference_ready:false` — alongside a Coral Edge TPU or an RKNPU. No engine PAIR runs has an Axera backend, so the row never counts toward GPU pressure and never marks `telemetryValid`.

The card has its own SoC, DRAM and operating system, so the host kernel exposes no counters for it. What is read:

| What | Source |
| ---- | ------ |
| Presence, one row per card | `/sys/bus/pci/devices/*` with vendor `0x1f4b`. Listed even when the vendor CLI is missing or failing; such a row is named from the PCI ids (`"Axera AX650N"`), carries `utilization_unavailable:true`, and has no memory or temperature. |
| Name | The card name `axcl-smi` prints plus the chip where known: `"Axera AX8850 (AX650N)"`. |
| `vram_bytes` / `vram_used_bytes` | The **CMM** pool from the `CMM-Usage` column — the contiguous memory NPU models and media buffers allocate from (2944 MiB on an AX8850). The card's `Memory-Usage` column is the card's own Linux RAM and is not reported. The pool belongs to the card, so there is no `memory_pool:"unified"`. |
| `utilization_percent` | The table's **NPU** column. Measured on an AX8850 running a YOLO11s model in a loop: 0 % idle, 89-93 % under load, back to 0 % when the run ended. `axcl-smi info --npu` is not used: it prints `NPU Usage (%) : --` at idle and a differently shaped `vnpu-Non : 92%%` line under load. A `--` in the column means no reading and sets `utilization_unavailable:true`; it is never published as 0. |
| `temperature_celsius` | The `Temp` field of the same table, whole degrees. |

One `axcl-smi` table call (about 0.6 s; it opens a session on the card) runs every **5 s** on its own goroutine, with a 5 s timeout and never overlapping the previous call, and one call is also made at startup so the first response already has numbers. A response uses the last good table only while it is at most 20 s old; after that the memory, utilization and temperature are dropped rather than frozen. A single failed call (the CLI returns no table if an operator runs a second `axcl-smi` at the same moment) is therefore invisible, and a warning is logged only after three consecutive failures. The CLI's own `file sink initialization failed` message on stderr is ignored.

**Power is a ceiling.** The card reports none (`Pwr:Usage/Cap` is `-- / --`), so `power_watts` is omitted, and there is no fan.

The CLI is expected at `/usr/bin/axcl/axcl-smi` (the AXCL package's install path) and must be runnable by the user the service runs as.

```json
{"name": "Axera AX8850 (AX650N)", "vram_bytes": 3087007744, "vram_used_bytes": 35651584, "utilization_percent": 91, "kind": "npu", "temperature_celsius": 37, "inference_ready": false}
```

## Shutdown

The service shuts down on:
Expand Down
Loading
Loading