[mlsorensen]

Summary

Adds signed community evidence for gb300-oke-ubuntu-training under gpuStack=oci-managed (GB300 NVL72, BM.GPU.GB300.4, OKE), the signer's allowlist entry, and the regenerated pointer index. This is the first OCI evidence in the repository.

Motivation / Context

Evidence for the OKE GB300 training recipe merged in #3706, generated from main after the merge. It replaces #3532, which was captured from the pre-merge recipe.

Fixes: N/A
Related: #3440

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Refactoring (no functional changes)
• Build/CI/tooling

Component(s) Affected

• CLI (cmd/aicr, pkg/cli)
• API server (cmd/aicrd, pkg/server)
• Recipe engine / data (pkg/recipe)
• Bundler and deployers (pkg/bundler, pkg/component)
• Collectors / snapshotter (pkg/collector, pkg/snapshotter)
• Validator (pkg/validator)
• Core libraries (pkg/errors, pkg/k8s)
• Docs/examples (docs/, examples/)
• Other: recipes/evidence/, pkg/evidence/index/pointers.yaml

Implementation Notes

• Source: generated from main at ae0da58180d27b7bed54631429ebf8b25668b465 (feat(recipes): add gb300-nvke training and inference leaves #3748), the first main commit after feat(recipes)!: add GB300 OKE recipe; harden OKE container toolkit #3706 with published validator images. The aicr CLI, recipe and bundle all come from that commit.
• Digest match: the evidence check reports the recipe digest as stale against the current main. The bundle attests to the recipe at ae0da5818; the only change since that reaches this recipe is feat(nvsentinel): add opt-in CSP Health Monitor mixins, STORE_ONLY #3687's added csp-health-monitor assertions in NVSentinel's health check, for a component off in every shipped recipe.
• Validators: ghcr.io/nvidia/aicr-validators/{deployment,conformance,performance}:sha-ae0da58180d27b7bed54631429ebf8b25668b465, published from main; no mutable tags.
• Snapshot agent: self-built from the same commit with ko (the repository's base image and user 65532) and pushed to ghcr.io/mlsorensen/aicr:main-ae0da5818@sha256:e09b045c9b6a1b326aecf9056811ab296a950929cf8a3987e1c34eac2bc17be4. No published agent image contains the K8s.oke-gpu-plugin and OS.kmod-param collectors yet: agent images are published at release tags, and v1.2.0 predates feat(recipes)!: add GB300 OKE recipe; harden OKE container toolkit #3706. oci-managed recipe resolution needs the oke-gpu-plugin reading.
• Cluster: 4× BM.GPU.GB300.4 on OKE, Kubernetes 1.36.4, Flannel pod networking, Oracle's 6.17-kernel GB300 image (driver 595.112.07, DOCA 3.5.0, RDMA subsystem in shared network namespace mode), NvidiaGpuPlugin add-on with deviceListStrategy=volume-mounts. Recipe hydrated from the snapshot with --intent training --profile gpuStack=oci-managed; bundle with no --set overrides, deployed with its deploy.sh.
• Bundle: ghcr.io/mlsorensen/aicr-evidence:gb300-oke-ubuntu-training-gpustack-oci-managed-95a59187166b (sha256:99b1833d…, a public package), keyless-signed (GitHub OAuth issuer), Rekor log index 193163255. The signer's new source slug is added to allowlist.yaml under community.
• Results: deployment, conformance and performance phases all pass; cluster-autoscaling is skipped (no Karpenter; OKE is not a recognized platform for that check). NCCL all-reduce bus bandwidth: NET 376.4 GB/s (floor 300), NVLS 909.3 GB/s (floor 700).

Testing

make qualify
go run ./tools/evidence-pointercheck -root recipes/evidence
go test ./pkg/evidence/... -count=1
aicr evidence verify recipes/evidence/gb300-oke-ubuntu-training-gpustack-oci-managed/a5460c0aead783a8b1316659e75e7acc/sha256-99b1833dcfe318f1b7f6385ecce100ac94e3ecfbc26fa3570ff2035f3794487b.yaml

make qualify passes; the pointer contract check is OK; evidence tests pass; evidence verify reports the bundle valid (signature, predicate, manifest hash, and recipe identity verified).

Risk Assessment

• Low — Isolated change, well-tested, easy to revert
• Medium — Touches multiple components or has broader impact
• High — Breaking change, affects critical paths, or complex rollout

Rollout notes: N/A

Checklist

• Tests pass locally (make test with -race)
• Linter passes (make lint)
• I did not skip/disable tests to make CI green
• I added/updated tests for new functionality
• I updated docs if user-facing behavior changed
• Changes follow existing patterns in the codebase
• Commits are signed off and cryptographically signed (git commit -s -S) — GPG signing info

[github-actions]

Recipe evidence check

Protected recipes

Recipes with committed evidence (recipes/evidence/<slug>/<source>/<digest>.yaml) that this PR affects: 1

Recipe	Source	Pointer	Verify	Digest match
gb300-oke-ubuntu-training-gpustack-oci-managed	a5460c0aead783a8b1316659e75e7acc	sha256-99b1833dcfe318f1b7f6385ecce100ac94e3ecfbc26fa3570ff2035f3794487b	✅ passed	⚠️ stale (5d582e1f1749… vs current 12e0bffa1c88…)

How to refresh evidence

Run on a cluster matching the recipe's criteria:

aicr snapshot -o snapshot.yaml
# Profiled families (AKS/GKE gpuStack): hydrate the recipe with the
# pointer's recorded 'profile:' selection first — validating the raw
# overlay resolves only the declaration default, and 'aicr validate'
# has no --profile flag. AKS additionally needs the pool projection
# (GKE uses the plain snapshot above):
#   az aks nodepool list -g <rg> --cluster-name <cluster> -o json > pools.json
#   aicr snapshot --aks-gpu-pools pools.json -o snapshot.yaml
#   aicr recipe -s snapshot.yaml --intent <intent> [--platform <platform>] \
#     --profile <name>=<value> -o recipe.yaml
# State the target leaf's intent/platform explicitly (the snapshot
# fingerprint supplies service/accelerator/OS but intent and platform
# default to 'any') and pass -r recipe.yaml below instead of the raw
# overlay.
aicr validate \
  -r recipes/overlays/<slug>.yaml \
  -s snapshot.yaml \
  --emit-attestation ./out \
  --push ghcr.io/<your-fork>/aicr-evidence
# Copy to the per-source path printed in the emit 'copyTo' hint:
#   recipes/evidence/<slug>/<source>/<bundle-digest>.yaml

This gate is warning-only and never blocks merge. See ADR-007 for the trust model.

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

• Source mode: Auto
• Repository: NVIDIA/aicr
• Branch: main
• File: .coderabbit.yaml
• Commit: ec5e033
• Layers: Repository YAML
• Review profile: ASSERTIVE
• Plan: Enterprise
• Run ID: 4f40776f-1da1-4687-a62d-6b0f5dfdbff2

Commits

Reviewing files that changed from the base of the PR and between 957e7cb and fcac96a.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The change adds an attestation record for the GB300 OKE OCI-managed recipe, a community signer entry, and an evidence index pointer that uses the same source slug.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Other

Possibly related issues

• Add GB300 OKE training recipe (BM.GPU.GB300.4) #3440: This change adds signed evidence for the GB300 OKE OCI-managed recipe named in the issue.

Possibly related PRs

• NVIDIA/aicr#3532: This PR publishes evidence for the same recipe.

Suggested labels: theme/supply-chain

Suggested reviewers: yuanchen8911

Merge Risk: Minimal · previous review 957e7

This change registers signed GB300 OKE evidence, its community signer, and the index pointer. The signer and index entries appear consistent with the attestation. One thing to confirm after merge is that the evidence-ingest registry trust setting includes the publishing registry, otherwise the dashboard may not pick up this evidence.

Security architecture risk: Low · up to 957e7

One additional community signer becomes eligible to influence recipe corroboration. Existing signature checks and distinct-signer counting remain in place, and no security defect was established. The referenced external signature was not independently checked.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — Control of the newly authorized signing identity could influence corroboration for any recipe whose admissible evidence reaches ingestion, not just GB300 OKE. The demonstrated authority is evidence weighting, and one identity alone cannot satisfy the two-signer CONFIRMED threshold.

Trust Boundaries and Controls

• observed — Structural admission checks bind the source directory to the signer-derived slug and the recipe directory to the pointer recipe. Cryptographic verification separately rejects signer identity, issuer, or Rekor-index mismatches. The external evidence was not fetched, so the specific committed signature claims remain unverified by this review.

Resilience and Maintainability Implications

• observed — Repeated evidence does not acquire additional identity weight: corroboration uses verified issuer/identity as the signer key and consensus defensively deduplicates that key. Missing or not-run results contribute no passing or failing weight.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly identifies the GB300 OKE training evidence change, which matches the main changeset.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The description explains the signed GB300 OKE evidence, signer allowlist entry, and regenerated pointer index. It is directly related to the changeset.

Finishing Touches

Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/aicr
• Branch: main
• File: .coderabbit.yaml
• Commit: aef688e
• Layers: Repository YAML
• Review profile: ASSERTIVE
• Plan: Enterprise
• Run ID: 8c7d984e-7f55-4a10-bb2c-3833878d7717

Commits

Reviewing files that changed from the base of the PR and between a644bea and 957e7cb.

Files selected for processing: 1

• recipes/evidence/gb300-oke-ubuntu-training-gpustack-oci-managed/a5460c0aead783a8b1316659e75e7acc/sha256-99b1833dcfe318f1b7f6385ecce100ac94e3ecfbc26fa3570ff2035f3794487b.yaml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[github-actions]

Coverage Report ✅

Metric	Value
Coverage	86.3%
Threshold	83%
Status	Pass

Coverage Badge

![Coverage](https://img.shields.io/badge/coverage-86.3%25-brightgreen)

No Go source files changed in this PR.

[njhensley]

Review — PR #3755 docs(evidence): GB300 OKE training (oci-managed)

Method: quick; 1 persona pass (correctness/API contract + evidence supply-chain trust), adjudicated by the parent in a separate adversarial pass against head a644bea. The later head 957e7cb only republishes the bundle to the author's public ghcr namespace and updates the pointer, which resolves the anonymous-pull (HTTP 401) warning the evidence check raised at a644bea. Only one reviewer was used, so this review has less independence than a full panel. Target was 10 minutes; it ran somewhat over.
Tier legend: 🔴 Blocker · 🟠 Major · 🟡 Minor · 🔵 Nitpick · ✅ Confirmed non-issue

Prior-feedback status

There are no human reviews. Bot feedback at the current head:

Disposition	Prior finding	Current evidence
Acknowledged	CodeRabbit: no actionable comments	—

Overall assessment — Approve

The PR adds one community evidence pointer for gb300-oke-ubuntu-training under gpuStack=oci-managed, the signer's allowlist entry keyed by source slug, and the regenerated index entry. The pointer, slug, allowlist entry and index are consistent with each other and with existing profiled pointers. No defects survived adjudication.

✅ Confirmed non-issues

• Source slug derivation. sha256("https://github.com/login/oauth\n1047709+mlsorensen@users.noreply.github.com")[:32] = a5460c0aead783a8b1316659e75e7acc. That matches the directory name, the allowlist.yaml entry and the pointers.yaml entry.
• OAuth issuer under community. Five existing community entries already use https://github.com/login/oauth. The allowlist rules only require slug-anchored or tightly bounded entries.
• Recipe id and profile. RecipeNameFor (pkg/evidence/attestation/builder.go) appends gpustack-<value> to the criteria-derived name. recipes/overlays/gb300-oke-ubuntu-training.yaml declares gpuStack=oci-managed.
• Pointer schema. It has the same fields and shape as the existing profiled pointers (schemaVersion: 1.0.0, profile: gpuStack=<value>, predicate recipe-evidence/v3, a single attestation).
• Index ordering. The new entry sits in sorted position between gb300-nvke-ubuntu-training-kubeflow and h100-aks-ubuntu-inference-dynamo, with class: community.
• Identity in the pointer file. It is a GitHub noreply address, as in the other OAuth pointers. The allowlist stays slug-only, with a non-PII label.
• Provenance claim. The source commit ae0da5818 cited in the PR description (#3748) is on main.

Coverage and limitations

• Inspected at the pinned head: all three changed paths, plus the allowlist header rules, the existing pointers in recipes/evidence/, the overlay, and the recipe-name builder.
• Executed: go run ./tools/evidence-pointercheck -root <PR tree>/recipes/evidence exited 0. This rules out path-ownership and allowlist-membership violations.
• Not executed:
  • aicr evidence verify against the bundle, because it cannot be pulled anonymously; the signature, Rekor entry and recipe identity are unverified here.
  • TestPointersYAMLMatchesTree on the PR tree; ordering was checked by hand instead.
• Taken from the PR description and not verified: the cluster facts, the NCCL numbers, and the claim that make qualify passed.

Summary

Tier	Count	Items
🔴 Blocker	0	—
🟠 Major	0	—
🟡 Minor	0	—
🔵 Nitpick	0	—

Recommendation: approve. Two things remain outside this PR. The NVIDIA/aicr-evidence package's internal visibility blocks anonymous verification for this pointer and four existing ones (see the bot check), and the branch is behind main and needs a rebase before it can merge.