From 8268334a2ba921a534abcca405722c05eaf17d83 Mon Sep 17 00:00:00 2001 From: ShivanshuPurohit Date: Thu, 16 Jul 2026 14:03:17 +0530 Subject: [PATCH] docs: refresh Optima architecture and operations --- docs/architecture/direct-artifacts.md | 358 ++++++++++++++ docs/architecture/overview.md | 5 +- docs/architecture/pipeline.md | 70 ++- docs/architecture/product-model.md | 16 +- docs/architecture/releases.md | 98 +++- docs/architecture/seam.md | 21 +- docs/architecture/slot-contract.md | 11 + docs/architecture/stacks.md | 33 +- docs/dev/gpu-setup.md | 37 +- docs/dev/sglang-tracking.md | 17 +- docs/engine/integration.md | 2 +- docs/engine/overview.md | 29 +- docs/engine/release-workflow.md | 47 +- docs/engine/verify-run.md | 81 ++- docs/get-started/concepts.md | 8 +- docs/get-started/quickstart.md | 6 +- docs/history/changelog.md | 82 +++- docs/index.md | 13 +- docs/miner-guide/bundle-format.md | 95 +++- docs/miner-guide/dep-patches.md | 15 +- docs/miner-guide/diagnostics.md | 13 +- docs/miner-guide/discovery-lane.md | 18 +- docs/miner-guide/examples.md | 18 +- docs/miner-guide/finding-a-win.md | 27 +- docs/miner-guide/graph-safety.md | 8 +- docs/miner-guide/kernel-abi.md | 65 +++ docs/miner-guide/override-points.md | 28 +- docs/miner-guide/overview.md | 15 +- docs/miner-guide/slots.md | 12 +- docs/miner-guide/submitting.md | 18 +- docs/miner-guide/your-first-kernel.md | 34 +- docs/reference/arenas.md | 13 +- docs/reference/cli.md | 219 ++++----- docs/reference/codebase-map.md | 62 ++- docs/reference/glossary.md | 36 +- docs/reference/manifest-schema.md | 130 +++++ docs/reference/slots-table.md | 13 +- docs/reference/stack-manifests.md | 9 +- docs/reference/state-of-record.md | 460 +++++++++--------- docs/reference/target-catalog.md | 10 +- docs/security/anti-copy.md | 5 +- docs/security/evidence.md | 140 ++++-- docs/security/isolation.md | 26 +- docs/security/threat-model.md | 8 +- docs/validator-guide/arena-service.md | 37 +- docs/validator-guide/chain-loop.md | 10 +- docs/validator-guide/fidelity.md | 31 +- docs/validator-guide/first-hour.md | 22 +- docs/validator-guide/overview.md | 15 +- docs/validator-guide/qualification.md | 81 ++- docs/validator-guide/releases.md | 64 ++- docs/validator-guide/running-evals.md | 233 +++++---- docs/validator-guide/scoring.md | 37 +- .../validator-guide/settlement-and-weights.md | 70 ++- docs/validator-guide/testnet.md | 54 +- mkdocs.yml | 3 +- 56 files changed, 2176 insertions(+), 912 deletions(-) create mode 100644 docs/architecture/direct-artifacts.md diff --git a/docs/architecture/direct-artifacts.md b/docs/architecture/direct-artifacts.md new file mode 100644 index 0000000..23d83fb --- /dev/null +++ b/docs/architecture/direct-artifacts.md @@ -0,0 +1,358 @@ +# Sealed direct artifacts + +Optima can execute a contribution as a sealed CUDA artifact without retaining a +miner-supplied Python launcher in the engine. The manifest describes a closed +launch program; a validator-owned builder compiles it to CUBIN in an isolated +prebuild; and validator-owned runtime code admits, materializes, and launches the +artifact. + +This path does not enlarge the slot boundary. The validator still owns semantic +inputs and outputs, allocation, streams, specialization selection, correctness, +graph evidence, qualification, and release admission. + +## Registered provider + +The provider registry is an immutable validator table. It has no manifest or +plugin registration API. The registered table contains one provider: + +| Property | Registered value | +|---|---| +| Provider ID | `cutlass.cute.cubin.v1` | +| Artifact kind | `cuda_cubin` | +| Binding ABI | `cuda_driver_params.v1` | +| Build phase | `oci_prebuild_only` | +| Load phase | `isolated_engine_worker_only` | +| Reviewed patcher | `optima.build-cute-cubin.v1` | +| Rebuild feature | `rebuild:build_cute_cubin` | +| Publication directory | `cute_cubin` | +| Qualification status | authoritative and crownable | + +A target must admit both `aot:cutlass.cute.cubin.v1` and +`rebuild:build_cute_cubin`. An unknown provider, an unregistered build feature, +or a provider that is not both authoritative and crownable is rejected before +evaluation or release materialization. The complete provider-registry snapshot +and digest are part of the target-catalog identity. + +The provider ID selects data-only policy. It does not select a compiler callback, +loader callback, host launcher, or candidate-defined ABI. Those implementations +remain reviewed Optima code. + +## Contract layers + +```mermaid +flowchart LR + M["Manifest declaration
exports, bindings, resources, device plan"] + I["Direct-execution identity
canonical declaration digest"] + P["Measured compile profile
image, hardware, topology, architecture"] + B["No-network/no-GPU prebuild
reviewed patcher + compiler child"] + C["Sealed CUBIN publication
complete index and file digests"] + R["Scheduler-rank admission
driver-observed ABI on retained handle"] + L["Validator launch runtime
parameters, resources, stream, lifecycle"] + E["Execution evidence
loaded + invoked + completed; no fallback"] + + M --> I + M --> B + P --> B + B --> C + C --> R + I --> R + R --> L + L --> E +``` + +Each layer is independently reopened. Matching source bytes are insufficient if +the provider policy, compile profile, device plan, native publication, observed +ABI, or runtime evidence differs. + +## Manifest execution model + +An operation uses `[[ops.aot_exports]]` to declare one or more ordered artifact +steps. Each export requires: + +- `provider`, `name`, and the compiler-side `factory` symbol; +- `profile_inputs`, selected from the provider's bounded allowlist; +- ordered semantic `bindings`; and +- a complete `device_plan` for the device provider. + +Optional `role`, `plan`, `step`, `specializes`, and `prelaunch` fields define a +bounded lifecycle and specialization program. `provider_capability_requirements` +and `specialization_capability_requirements` are validator-derived when a +manifest is parsed and must reproduce exactly when sealed state is reopened. + +`factory` names Python executed only in the disposable compiler child. It is not +an engine entry point. The compatibility `ops.entry` field remains syntactically required +by the bundle ABI, but direct-artifact execution does not call it and canonicalizes +the runtime entry to `_optima_direct_artifact`. + +Plan variants use exact-equality specialization predicates. Overlap is accepted +only when it forms a strict fallback chain. Runtime chooses the unique +most-specific matching plan; an ambiguous or missing required direct binding is +an error rather than a request to execute candidate host code. + +See [Bundle manifest](../reference/manifest-schema.md) for the field-level schema. + +## Compile profile and architecture + +CuTe compilation can require hardware facts even though prebuild has no GPU. A +trusted arena measures those facts first and writes a canonical +`optima.native-cute-compile-profile.v1` document. The profile binds: + +- logical architecture, such as `sm103`; +- compiler architecture, such as `sm_103a`; +- image, platform, worker-distribution, logical-hardware, device-policy, and + topology digests; +- visible GPU count and TP, EP, and DP degrees; +- bounded provider constants and their measurement digest; and +- the provider and schema. + +Logical and compiler architectures must name the same architecture family. The +profile is mounted read-only into prebuild and is checked again against the launch. +It is data only: it imports neither CUDA nor candidate code. + +The registered provider permits only these profile inputs: + +```text +max_active_clusters.cluster_size_1 +max_active_clusters.cluster_size_2 +max_active_clusters.cluster_size_4 +max_active_clusters.cluster_size_8 +max_active_clusters.cluster_size_16 +``` + +An export selects the only profile values its factory can receive. Candidate code cannot +probe the device or request a constant outside that declaration; the parser does not +attempt to prove that factory code uses every selected value. A CUBIN compiled under one profile +is not portable authority for a launch with a different logical architecture, +image, topology, device policy, or profile digest. + +## Hermetic prebuild + +Native compilation happens before an engine arm starts. The controller launches a +disposable OCI container with: + +- a digest-pinned image and platform with `--pull=never`; +- `--network=none`, no GPU visibility, and no model, wallet, home, Docker socket, + or ambient host mount; +- a read-only root and engine-tree mount; +- a read-only compile-profile mount and one lease-scoped output mount; +- all Linux capabilities dropped, `no-new-privileges`, reviewed seccomp, a + non-root UID/GID, and CPU, memory, PID, tmpfs, time, inode, and output quotas; +- `CUDA_VISIBLE_DEVICES=""` and `NVIDIA_VISIBLE_DEVICES=void`. + +The outer worker parses candidate files as data and invokes only the registered +rebuild patcher. The patcher starts a compiler child, imports the validator's CuTe +compiler and options before importing the candidate module, and calls the declared +factory with a bounded profile context. The factory must return the exact +validator-defined compile-request type. The validator forces the no-JIT-engine +mode and its compile options, including CUBIN retention. + +The authoritative output is CUBIN only. Generated host objects, Python launchers, +PTX fallbacks, and JIT engines are not published. The host never imports candidate +Python and never loads the produced native object. + +## Sealed publication + +The prebuild writes a bounded canonical index plus the exact CUBIN files. A +GPU-free reopen verifies the directory inventory, file paths, sizes, SHA-256 +digests, and CUDA ELF header without importing the CUDA driver. The index binds: + +- provider and binding ABI; +- engine-tree, native-build-spec, compile-profile, and patcher digests; +- logical and compiler architecture; +- installed CuTe/CUTLASS distribution identities and forced compile options; +- export, target-authority, resource-plan, binding, specialization, prelaunch, + and device-plan declarations; and +- every CUBIN's bytes and complete logical kernel contract. + +The runtime mount is read-only. A scheduler rank can reopen and load the +publication but cannot compile a missing artifact or repair a mismatched index. + +## Complete device ABI + +`device_plan` uses schema `optima.device-launch-plan.v1`. It contains two closed +inventories: + +1. `kernels`: sorted, unique logical kernel names and the exact byte width of every + formal parameter; +2. `launches`: a contiguous ordinal sequence naming a logical kernel, grid, block, + optional cluster, dynamic shared-memory expression, ordered parameters, stream + binding, and allowlisted launch attributes. + +The plan schema admits at most 1,024 logical kernels, 256 launches, and 64 +semantic bindings. Dynamic shared memory is an expression over live call data; +after evaluating it for an invocation, the runtime rejects values above 1 MiB. +A launch's parameter widths must equal its logical kernel contract exactly. + +CuTe-generated physical symbol names depend on the materialized module name, so +they are not a stable miner-facing identifier. After the scheduler rank has +completed CUDA device setup, the loader opens the exact CUBIN, observes its full +physical kernel inventory through the CUDA Driver API, and compares the complete +per-ordinal parameter-width vectors. Canonical logical ordinal is then bound to +physical ordinal. The admitted library handle is retained for launch; Optima does +not inspect, unload, and reopen a second object. + +Admission computes the CUBIN digest and size plus driver-observed ABI and contract +digests. The driver walk observes physical names and parameter offsets/sizes to +derive that ABI digest, but the retained load receipt stores the digests rather +than the complete raw physical inventory. Logical-to-physical selection remains +ordinal. + +## Validator-owned materialization + +The device plan is a data program. At each invocation Optima constructs the CUDA +parameter frame from the live slot call. Its closed parameter primitives are: + +| Primitive | Validator action | +|---|---| +| `pointer` | Projects a device pointer from an admitted semantic binding | +| `scalar` | Evaluates a checked expression and packs an exact scalar type | +| `packed_struct` | Zeroes the ABI frame, then writes declared scalar, pointer, or bounded static fields at checked offsets | +| `tma_descriptor` | Encodes a versioned 128-byte tensor map from an admitted tensor and checked dimension expressions | +| `cutlass_fast_divmod_i32_v1` | Constructs the fixed 12-byte CUTLASS FastDivmod representation | +| `cute_fast_divmod_i32_v1` | Constructs the fixed 12-byte CuTe FastDivmod representation | +| `group_handle` | Projects a provider-authorized native handle or peer-pointer table | + +Checked expressions can use bounded arithmetic over live tensor dimensions, +strides, element counts, and admitted scalars. Candidate pointer construction, +arbitrary integer addresses, callbacks, host objects, implicit Python +conversions, and unregistered expression operations are absent. Grid, block, +optional cluster, shared memory, parameters, and stream are all materialized by +validator code. Launch uses `cuLaunchKernelEx` on an exact captured stream handle +when one is declared, or on a validator-constructed default-stream handle. + +## Artifact-owned storage and lifecycle + +`[[ops.artifact_resources]]` declares storage that Optima allocates. A declaration +names a dtype, power-of-two alignment, bounded shape, lifetime, and optional +scope. The prefix fixes the lifetime: + +| Name prefix | Required lifetime | Meaning | +|---|---|---| +| `workspace.*` | `call` | Logical call-local scratch; uninitialized storage may be cached and reused for the same plan/execution/group/shape key | +| `prepared.*` | `prepared` | Produced by `prepare`, then consumed by `run` | +| `state.*` | `engine` | Persistent state owned by one engine artifact entry | + +The default scope is `rank_local`. `group_ipc` is declaratively available only +for persistent `prepared` or `engine` resources; call-local group IPC is rejected. +Shapes use only `ceil(product(factors) / divisor)`, where a factor is a positive +constant or a bounded projection of an admitted tensor, integer scalar, or group. + +Manifest bounds are at most 32 resources, rank 1–16, 1 MiB alignment, 64 GiB per +buffer, and 128 GiB aggregate maximum size. These are syntax/admission ceilings, +not promised allocations. The current CUBIN runtime applies the lower shared +budget of 16 GiB live storage and 4,096 allocation keys. + +Exports have ordered roles `init`, `prepare`, `reset`, `run`, and `destroy`. +Validation enforces resource access at those boundaries: workspace cannot appear +in init or destroy; prepared storage must be produced in prepare and consumed in +run; and one destroy export cannot mix lifetime/scope classes. Prelaunch is also +closed: the registered operation is a bounded validator-executed `fill` of an +authorized target. + +Storage declarations cannot mint a stream, process group, arbitrary pointer, or +new semantic input/output. The immutable slot call ABI remains the resource root. + +## Runtime selection and evidence + +Runtime preparation occurs in each isolated scheduler rank after CUDA has selected +that rank's device. It reopens the native publication and manifest, requires exact +declaration equality, constructs the validator primitive registry, admits every +CUBIN, initializes lazy lifecycle state, and prebinds exactly one runtime entry +per `(slot, variant)`. Artifact storage is allocated on the applicable prepare or +call path and may be cached under its admitted key. A process cannot be rebound to +a different direct-artifact authority. + +Selection is fail-closed: + +- all exports for a direct operation must belong to the one registered provider; +- the provider runtime must already be prepared; +- the exact `(slot, variant)` binding and unique specialization plan must exist; +- a driver-observed CUBIN/ABI mismatch rejects and unloads the handle; +- launch or cleanup uncertainty terminates the isolated worker rather than + authorizing a result. + +Execution authority requires three evidence layers: + +| Receipt | Emission point | +|---|---| +| `aot_loaded` | All exports for the slot/variant were reopened, admitted, and bound | +| `aot_invoked` | The validator-generated entry returned successfully at least once | +| `completed` | The normal registered seam completed for the slot on that member | + +The completion gate requires identical active slot coverage across all scheduler +members, full per-member AOT load and invocation coverage for AOT slots, and zero +fallback receipts. An invocation without a load is invalid. A loaded artifact +that never executes cannot qualify as the measured contribution. + +## Identity and copy signals + +Direct execution has canonical schema `optima.direct-artifact-execution.v1`. +Its identity covers the resource plan and every execution-bearing export field: +provider, factory, profile inputs, role, plan, step, ordered bindings, +specializations, prelaunch operations, derived capability requirements, and the +complete device plan. It deliberately excludes the unused compatibility `ops.entry`. +Finite floating-point values are type-tagged and represented exactly. + +That execution identity is included in the selected contribution identity, +selected-delta digest, stack materialization, and engine-tree digest. The native +build specification, compile profile, sealed publication, and runtime admission +add later identities; none is substituted by a friendly export or provider name. + +The bundle content hash separately covers every identity-bearing regular file by +relative path and bytes. Copy/provenance analysis uses a normalized per-slot +fingerprint that combines the canonical direct-execution declaration with the +normalized transitive Python source closure. Semantically relevant declaration +changes remain in this signal; formatting, TOML row order, and the unused +compatibility entry are intentionally canonicalized away. Exact path and byte +identity remain available through the ordinary bundle and selected-payload digests. + +## Support boundary + +The following are contract boundaries, not implementation suggestions: + +- Only `cutlass.cute.cubin.v1` is registered. A bundle cannot add a provider. +- The only artifact and binding pair is CUDA CUBIN with + `cuda_driver_params.v1`; there is no miner host launcher. +- Candidate factory Python executes during isolated prebuild, never as the runtime + launch adapter. +- The launch, expression, parameter, resource, lifecycle, and attribute languages + are closed and bounded. A genuinely new primitive or semantic boundary requires + validator code and target-catalog policy. +- Compile profiles and sealed artifacts are exact to their measured launch + authority; architecture-family compatibility alone does not make them reusable. +- Release identity can bind a direct-artifact native build and publication. The + serving environment must derive `OPTIMA_CUTE_COMPILE_PROFILE_DIGEST` from that + signed build specification and propagate it unchanged. Missing propagation or + missing direct-artifact serve-smoke authority must fail closed; identity support + alone cannot authorize serving. +- The provider and schemas declare native-group-handle and peer-pointer-table + vocabulary. The standard `build_cute_cubin` load path does not install group + capability/handle resolvers, so plans requiring those projections fail closed + before execution. This is an implementation support gap in addition to the lack + of empirical qualification evidence. + +Hardware evidence should therefore be stated per concrete slot, topology, plan, +and artifact identity. The direct-artifact contract is general enough to express +rank-local and group-aware plans, but the existence of schema vocabulary does not +make the standard runtime capable of resolving a group-aware plan or promote an +unmeasured collective into a proven production path. + +Empirical coverage is recorded per concrete provider, slot, topology, plan, and +artifact identity in the [State of record](../reference/state-of-record.md). +Evidence from a group-aware schema, another direct-artifact plan, or a separate +hardened native-build lane does not transfer qualification to an unmeasured +provider/slot combination. + +## Source map + +- [`artifact_provider.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_provider.py) — closed provider registry and profile-input policy +- [`manifest.py`](https://github.com/latent-to/optima/blob/main/optima/manifest.py) — export parsing and target admission +- [`artifact_abi.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_abi.py) — semantic bindings, storage, and lifecycle +- [`artifact_device_launch.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_device_launch.py) — device-plan schema, admission, and runtime +- [`cuda_materialize.py`](https://github.com/latent-to/optima/blob/main/optima/cuda_materialize.py) — checked parameter and TMA materialization +- [`cuda_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/cuda_cubin.py) — Driver API observation and retained-handle admission +- [`cute_aot.py`](https://github.com/latent-to/optima/blob/main/optima/cute_aot.py) — compiler-factory boundary and invocation evidence +- [`cute_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/cute_cubin.py) — sealed index and runtime binding +- [`eval/native_compile_profile.py`](https://github.com/latent-to/optima/blob/main/optima/eval/native_compile_profile.py) — measured GPU-free compile inputs +- [`eval/oci_prebuild.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_prebuild.py) — hermetic native prebuild +- [`artifact_identity.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_identity.py) — canonical direct-execution identity diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 4e16c76..8cbf97b 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -115,7 +115,10 @@ The authoritative path is: 7. journaled reward projection and weight publication; 8. independent integration review and signed engine release. -Local `verify`, `evaluate`, and `bench` commands are developer diagnostics. They do not mint production crown authority. See [Evaluation pipeline](pipeline.md) for the detailed state machine and [Releases](releases.md) for the ship boundary. +`scan` and `verify` are contributor diagnostics. Matched A/B profiling on +contributor-controlled hardware may test a performance mechanism, but none of these paths +can mint production crown authority. See [Evaluation pipeline](pipeline.md) for the +detailed state machine and [Releases](releases.md) for the ship boundary. ## Design acceptance tests diff --git a/docs/architecture/pipeline.md b/docs/architecture/pipeline.md index 578c033..b13a166 100644 --- a/docs/architecture/pipeline.md +++ b/docs/architecture/pipeline.md @@ -10,7 +10,7 @@ The developer commands intentionally reuse parts of the ABI and evaluation machi ## Developer path ```text -slots -> scan -> verify -> evaluate / bench -> chain-package -> host -> chain-submit -> chain-status +slots -> scan -> verify -> chain-package -> host -> chain-submit -> chain-status ``` | Step | Purpose | Authority | @@ -18,8 +18,6 @@ slots -> scan -> verify -> evaluate / bench -> chain-package -> host -> chain-su | `slots` | Inspect the live slot ABI | Informational | | `scan` | Run static bundle policy checks | Diagnostic | | `verify` | Check a single op/block against its trusted reference; collective slots use distributed verification | Diagnostic | -| `evaluate` | Run a narrow model-level development comparison | Diagnostic | -| `bench` | Run benchmark-oriented throughput and quality diagnostics | Diagnostic | | `chain-package` | Produce the deterministic hosted archive and content identity | Submission preparation | | HTTPS host | Make the immutable archive available for validator fetch | Transport only | | `chain-submit` | Commit the proposal through timelock commit-reveal | Chain intake | @@ -44,8 +42,12 @@ flowchart TD R -->|"attributable FAIL"| F R -->|"NO_DECISION"| N R -->|"matching PASS"| S["Reopen both evidence roots"] - S --> T["Transactional settlement and stack update"] + S --> G["Same-authority cohort planning"] + G -->|"selected current registered winner"| T["Transactional settlement and stack update"] + G -->|"current discovery candidate"| J["DISCOVERY_BOUNTY / no stack transition"] + G -->|"stale or not selected"| H["HOLD / no stack transition"] T --> W["Reward projection and journaled weights"] + J --> W T -. "separate decision" .-> I["Integration review and signed release"] ``` @@ -150,6 +152,14 @@ runtime ranks may validate and load native products but may never compile or repair them. Both stages use read-only roots, bounded mounts and protocols, and host-owned cleanup; the trusted controller also owns timing. +For a direct-artifact row, prebuild executes the declared compiler factory only +inside a no-egress compiler child and publishes CUBIN rather than a host launcher. +After rank-local CUDA setup, the scheduler worker admits the exact CUBIN, binds its +complete driver-observed ABI to the declarative device plan by ordinal, and +materializes parameters and lifecycle storage in validator code. Qualification +requires per-member `aot_loaded`, `aot_invoked`, and normal `completed` coverage, +with no fallback receipt. See [Sealed direct artifacts](direct-artifacts.md). + After C is destroyed, T grades its sealed trajectory under a separate pristine lifetime. T never contains the candidate and does not compete on speed. Hidden reference work, quality policy, and selected prompt identity are bound into retained evidence. The host pairs candidate throughput with the B/B′ bookend and applies the registered noise policy. Conceptually: @@ -196,10 +206,13 @@ Consider a hypothetical candidate for one registered singleton target: 4. A second independently selected authority repeats the exact reproduction identity and also passes. A pass against a newer incumbent or different target specification would not count as this reproduction. -5. Settlement reopens both evidence roots, takes the lower accepted speedup, revalidates - the live target transition, and atomically updates the evaluation stack. -6. Reward projection can now see the active claim. Product integration remains a separate - review; no proposal bytes have entered a release merely because settlement completed. +5. Settlement reopens both evidence roots and makes the pair eligible for its + same-authority cohort. If the candidate is selected as that cohort's current registered + winner, settlement takes the lower accepted speedup, revalidates the live target + transition, and atomically updates the evaluation stack; otherwise the pair is held. +6. If crowned, reward projection can now see the active claim. Product integration remains + a separate review; no proposal bytes have entered a release merely because settlement + completed. The values and candidate in this walkthrough are illustrative. The state transitions and failure semantics are the important part. @@ -225,16 +238,26 @@ the retained first pass cannot update the incumbent by itself. ## 9. Settlement -Settlement reopens both evidence roots instead of trusting an in-memory verdict. It verifies: +Settlement reopens both recorded attempt references instead of trusting an in-memory +verdict. The references may live under the same content-addressed store root. It verifies: - both reports are complete `PASS` results; -- authority and evidence roots are distinct; +- all seven required authority, attempt, report, commitment, and selection digests are + pairwise distinct across the two passes; - reproduction identities match exactly; - the incumbent and target transition are still current; - target overlap, displacement, and composition remain valid; - the requested stack update matches the measured candidate. -The conservative settled speedup is the lower accepted speedup from the two passes. The stack transition and settlement evidence are committed transactionally; a partial write cannot expose a half-updated incumbent. +The planner leases one cohort whose rows share qualification authority and incumbent +state. Stale rows are held, and exactly one current registered winner is selected across +the remaining rows, including rows whose targets do not overlap. Other current pairs are +held as `conflict_lost` or `incumbent_advanced`; a stale pair is held as +`stale_incumbent`. + +For the selected winner, the conservative settled speedup is the lower accepted speedup +from the two passes. The stack transition and settlement evidence are committed +transactionally; a partial write cannot expose a half-updated incumbent. Principal code: [`settlement.py`](https://github.com/latent-to/optima/blob/main/optima/settlement.py). @@ -259,7 +282,9 @@ Principal code: [`economics.py`](https://github.com/latent-to/optima/blob/main/o A settled crown may enter integration review, but serving remains a separate state machine. Reviewed source is promoted to an integrated contribution, assembled into an `EngineReleaseManifest`, rematerialized, signed, published, and verified without chain dependencies. -See [Release architecture](releases.md). Current implementation and validation limits are tracked in [State of record](../reference/state-of-record.md). +See [Release architecture](releases.md). The dated +[State of record](../reference/state-of-record.md) tracks implementation and validation +limits. ## Operational handoff checklist @@ -270,20 +295,17 @@ able to reopen, rather than merely observe, each handoff: publication; - registered arena, target catalog, incumbent manifest, candidate transition, and screen receipt; -- B/C/B′ launch identities, raw bounded session evidence, device/cleanup receipts, and - the candidate-free T evidence; -- frozen calibration and the exact policy that maps the raw evidence to the verdict; -- distinct primary and reproduction authorities over one reproduction identity; +- B/C/B′ launch identities and aggregate `SpeedWitness` rows, plus the retained + graph/quality/pristine-T references and witnesses; raw B/C/B′ session/device frames are + validated in-run but are not serialized into the attempt; +- frozen calibration and the exact policy that maps the retained witness/evidence products + to the verdict; +- the seven digest-distinctness fields across primary and reproduction over one + reproduction identity; - transactional settlement events and resulting evaluation-stack digest; -- reward projection, publication intent, chain submission/readback, and confirmed or held - state; and +- reward projection plus publication intent/status/chronology records; later readback + vectors must be re-observed because the journal does not serialize them; and - if shipping is proposed, the separate integration records and signed release identity. Console output, a green local benchmark, one `PASS`, or a successful chain SDK return is not a substitute for the corresponding reopenable product. - -## Legacy simulation is not production authority - -`commit`, `reveal`, `ledger`, and `legacy-settle --development-only` exercise an older local mechanism simulator. They do not write production SQLite qualification, settlement, or weight state. Production operator flow uses `chain-validate` and `set-weights`. - -The distinction matters when reading old examples or historical receipts: a successful legacy flow does not demonstrate the current isolated two-pass production path. diff --git a/docs/architecture/product-model.md b/docs/architecture/product-model.md index 12a1b36..493f250 100644 --- a/docs/architecture/product-model.md +++ b/docs/architecture/product-model.md @@ -1,6 +1,8 @@ # Product model -Optima Engine is a chain-independent, open inference-acceleration distribution. It pins an upstream runtime, initially SGLang, and assembles accepted data-plane optimizations into a canonical engine stack. +Optima Engine is a chain-independent, open inference-acceleration distribution. It pins +the supported SGLang runtime as part of product identity and assembles accepted data-plane +optimizations into a canonical engine stack. The subnet is a market around that product. It discovers, measures, attributes, and rewards improvements. It is not embedded in the released engine, and the managed service never serves mutable miner bundles or reads live chain state to decide what code to execute. @@ -123,7 +125,7 @@ The validator-owned target catalog defines what can receive ordinary attribution Miner packaging and manifest row order do not define economic scope. A bundle that explicitly claims a registered target but does not resolve to its exact members and allowed features fails resolution rather than falling through to an unregistered identity. -The catalog currently registers all singleton slots and the atomic `collective.moe_epilogue.v1` target. The atomic target owns both `collective.ar_residual_rmsnorm` and `collective.moe_finalize_ar_rmsnorm`, and explicitly displaces the corresponding singleton titles while active. The live policy is implemented in [`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py). +The registered catalog contains every singleton slot and the atomic `collective.moe_epilogue.v1` target. The atomic target owns both `collective.ar_residual_rmsnorm` and `collective.moe_finalize_ar_rmsnorm`, and explicitly displaces the corresponding singleton targets while active. The live policy is implemented in [`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py). ## Discovery lane @@ -180,7 +182,9 @@ The product model is intact only if all six statements hold: 1. Removing chain access and miner hosting does not prevent rebuilding or serving the latest signed release. 2. A new component is evaluated as one marginal substitution over the current stack. -3. The trusted controller never imports candidate code; complete engines remain the isolation unit. +3. The trusted controller never imports candidate code; candidate runtime execution stays + inside a complete isolated engine, while a sealed-direct-artifact factory may execute + only inside its further isolated no-network/no-GPU compiler child. 4. A whole-system prototype cannot acquire a duplicate permanent whole-engine reward title by packaging alone. 5. Every shipped component resolves to reviewed source and immutable attribution. 6. Updating the evaluation incumbent and publishing a release are independent, explicitly authorized state transitions. @@ -193,8 +197,10 @@ Before extending Optima, locate the feature in this model: discovery identity. “The whole engine” is not an acceptable default. 2. **Who supplies surrounding code?** The validator must assemble the incumbent; a miner must not be required to redistribute other contributors' bundles. -3. **Where does hostile code execute?** Proposal code belongs only in the complete isolated - evaluation engine, never in the controller or serving release by reference. +3. **Where does hostile code execute?** Runtime proposal code belongs only in the complete + isolated evaluation engine. A direct-artifact factory may run only in the bounded + compiler child during disposable prebuild. Neither path may import candidate code into + the controller or carry a miner runtime callback into a serving release. 4. **What creates product authority?** Identify the review record, preserved selected payload, integrated source, maintained tests, and release decision. 5. **Can the resulting release stand alone?** Rebuild, verification, and serving must not diff --git a/docs/architecture/releases.md b/docs/architecture/releases.md index f40591e..f21de6b 100644 --- a/docs/architecture/releases.md +++ b/docs/architecture/releases.md @@ -9,14 +9,24 @@ flowchart LR R --> M["EngineReleaseManifest"] M --> T["Materialized reviewed engine tree"] T --> P["Prepared release descriptor + artifacts"] - P --> S["Ed25519 signature
trusted by external expected key"] + P --> W["Clean-wheel closure +
provider-specific native proof"] + W --> S["Ed25519 signature
trusted by external expected key"] S --> U["Immutable release publication"] - U --> O["Two reproducible registry builds"] - O --> A["Authorized digest-pinned container"] + U --> O["Two builder-authenticated
reproducible registry outputs"] + O --> A["Authorized image + closed
effective argv/routes"] A --> V["Fail-closed serving runtime"] + V --> Q["Release/session-bound smoke
+ AOT coverage when applicable"] ``` -The current implementation contract is described here. Evidence about what has been exercised on real hardware is kept separately in [State of record](../reference/state-of-record.md). +This page defines the release contract. Evidence about which gates the pinned +implementation and retained hardware products satisfy is kept separately in +[State of record](../reference/state-of-record.md). + +!!! danger "Every arrow is required" + Registry digest equality, a signed descriptor, or a responsive canary cannot skip the + wheel, native-provider, builder-provenance, effective-policy, management-route, or + bound-receipt gates. The canonical executable checklist is + [Build and sign a release](../engine/release-workflow.md#publication-checklist). ## Promotion from crown to source @@ -83,6 +93,31 @@ Native products are prebuilt under a typed `NativeBuildSpec` and published into Scheduler ranks may load and validate sealed native products. They may not compile or repair a missing product during serving. The release publication carries the exact native address space expected by its descriptor. +For `cutlass.cute.cubin.v1`, the release build pipeline must produce the native input from +the release-rematerialized reviewed tree through the same registered, no-GPU/no-network +prebuild boundary used by evaluation. `prepare_release()` neither runs that build nor +provider-specifically validates the CuTe index/profile. It generically reopens the +caller-supplied `NativeArtifactPublication` inventory and binds its `NativeBuildSpec` and +publication digest. Release admission must therefore retain and independently reopen the +provider-specific prebuild, compile-profile, canonical index, and CUBIN proof before +calling the generic preparation API. + +The CuTe build profile covers logical and compiler architecture, image/platform, worker +distribution, device policy, topology, and validator-measured constants. Its publication +contains only sealed CUBINs and a canonical index; it does not contain a miner Python +launcher, host object, PTX fallback, or JIT engine. + +The direct-artifact load contract requires each scheduler rank, after CUDA device +setup, to reopen the exact publication, compare the complete per-ordinal parameter +widths with the driver-observed CUBIN inventory, retain the admitted handle, and +construct parameters and lifecycle resources from the signed manifest. The closed +serving environment must derive the CuTe compile-profile digest from the signed native +build specification and propagate it unchanged into that load phase. If that handoff is +absent or mismatched, a direct-CUBIN release must fail before startup. Release identity +support alone is not serving support. See +[Sealed direct artifacts](direct-artifacts.md) and consult the +[State of record](../reference/state-of-record.md) for implementation status. + Principal code: [`eval/native_artifact.py`](https://github.com/latent-to/optima/blob/main/optima/eval/native_artifact.py) and [`eval/engine_launch.py`](https://github.com/latent-to/optima/blob/main/optima/eval/engine_launch.py). ## Prepared release @@ -135,7 +170,7 @@ The publication is self-contained with respect to chain state. It retains proven - a deployment-policy document; - a generated Dockerfile. -The Dockerfile installs the deterministic serving wheel, installs only reviewed runtime overlays for the pinned upstream revision, copies the release under `/optima`, and uses `optima.release_runtime` as its entry point. The generated deployment policy requires a read-only root filesystem, a bounded writable `/tmp`, the reviewed seccomp profile, an exact model mount, and a receipt directory. +The Dockerfile installs the deterministic serving wheel, installs only reviewed runtime overlays for the pinned upstream revision, copies the release under `/optima`, and uses `optima.release_runtime` as its entry point. The generated deployment policy requires a read-only root filesystem, a bounded writable `/tmp`, the reviewed seccomp profile, an exact model mount, and a receipt directory. Context generation does not prove that the emitted wheel's complete import graph works; release acceptance installs it in a clean environment and exercises every manifest-reachable runtime entrypoint. No chain credentials or private signing key enter the context. @@ -149,9 +184,9 @@ No chain credentials or private signing key enter the context. - distinct temporary registry tags; - BuildKit-generated provenance/SBOM disabled in favor of Optima's signed canonical artifacts. -After each push, Optima reopens the Registry v2 manifest and configuration rather than trusting local image output. It verifies descriptor, seccomp, and reviewed-overlay labels. The two raw manifest digests must match. +After each push, Optima reopens the Registry v2 manifest and configuration rather than trusting a mutable tag. It verifies descriptor, seccomp, and reviewed-overlay labels. The two raw manifest digests must match. On equality, the current primitive signs the common descriptor/image/platform statement with Ed25519 as `SignedContainerReproducibility`; it does not retain or authenticate the builder invocations' output records. -The common descriptor/image/platform statement is then signed with Ed25519 as `SignedContainerReproducibility`. Deployment authorizes an image only after checking that attestation against the expected public key, release descriptor, platform, repository, and digest. +Canonical release authority therefore adds a separate provenance gate: each reopened registry object must be bound to the output emitted by its corresponding builder invocation before the signed reproducibility statement is accepted as authoritative. Tag readback plus digest equality alone does not provide that binding. Deployment then checks the accepted attestation against the expected public key, release descriptor, platform, repository, and digest. Principal code: [`release_host.py`](https://github.com/latent-to/optima/blob/main/optima/release_host.py). @@ -159,7 +194,7 @@ Principal code: [`release_host.py`](https://github.com/latent-to/optima/blob/mai The host launches by immutable `repository@sha256:`, never by a mutable tag. Before a container is accepted, host-side authorization reopens registry metadata and verifies the signed reproducibility statement. -Container creation and inspection enforce the deployment contract, including: +The implemented container-creation and inspection primitives enforce: - exact image digest and labels; - read-only root filesystem; @@ -167,10 +202,19 @@ Container creation and inspection enforce the deployment contract, including: - reviewed seccomp profile; - expected GPU device request; - exact read-only model bind mount; -- no unexpected mounts, privileges, or host namespace changes; -- signed entry point, command, and environment. - -The result is an `AuthorizedReleaseContainer`, not merely a successful `docker run`. +- no additional mounts, privilege escalation, or Linux capabilities; +- exact host network and IPC modes; and +- the registry image's exact entry point, command, and environment. + +The typed host primitive returns an `AuthorizedReleaseContainer`. Canonical production +authority combines that result with deployment-owned gates that close the effective parsed +SGLang argument vector after aliases, abbreviations, duplicates, loader/config indirection, +and defaults; bind that effective image configuration to the signed `ServeSpec`; close any +remaining PID, UTS, user, and cgroup namespace policy; and authenticate or disable +management routes exposed by the signed network mode. Builder-output provenance must also +have passed. Those external gates are part of the release contract but are not implemented +by the container primitive; an `AuthorizedReleaseContainer` is therefore never equivalent +to successful production authorization or merely to a successful `docker run`. ## Fail-closed runtime entry @@ -184,7 +228,13 @@ Inside the container, `verify_serving_release()` reopens the signed release befo - seam bindings derived from release slots; - native and engine-tree identities. -It then constructs a closed serving environment. `OPTIMA_RELEASE_REQUIRED=1` changes seam behavior from development fallback to fail-closed startup: missing namespace binding, seam installation, candidate activation, or registered slots terminates the process. The runtime also sets strict candidate execution and disables user-site/import-path escape hatches. +It must then construct a closed serving environment, including every signed native-profile +value required by sealed direct artifacts. `OPTIMA_RELEASE_REQUIRED=1` changes seam behavior +from development fallback to fail-closed startup: missing namespace binding, seam +installation, candidate activation, registered slots, compile-profile authority, or other +required runtime identity terminates the process. The runtime also sets strict candidate +execution and disables user-site/import-path escape hatches. Environment construction does +not replace effective-argv or management-route verification at the host/deployment boundary. Only after verification does the entry point `exec` the exact signed `sglang.launch_server` command. @@ -192,9 +242,25 @@ Principal code: [`release_runtime.py`](https://github.com/latent-to/optima/blob/ ## Serve smoke and receipts -A release smoke must produce `active`, `fired`, and `completed` seam coverage for every expected release slot and tensor-parallel rank, with no `load_failed` or `fallback` receipt. `verify_serve_receipts()` validates the fresh receipt directory and binds its result to the release descriptor. +A release smoke must produce `active`, `fired`, and `completed` seam coverage for every +expected release slot and tensor-parallel rank, with no `load_failed` or `fallback` +receipt. `verify_serve_receipts()` enforces that coverage over the supplied directory and +places the reopened release descriptor digest in its returned summary. + +The input frames do not themselves bind a unique release descriptor and serve-session +identity. The summary's descriptor field therefore cannot authenticate which process +emitted those files. Production acceptance requires descriptor/session binding at receipt +emission and verification; a directory with matching slot/rank names is insufficient. + +Direct-artifact loading and use emit `aot_loaded` and `aot_invoked`. Authoritative +qualification enforces full per-member coverage for those receipt classes, but +`verify_serve_receipts()` does not consume them. A release containing a sealed direct +artifact requires equivalent release-smoke AOT coverage, bound to the same release and +session, before it can be authorized. -This proves that the signed engine tree routed through its intended seams during the smoke. It does not replace the earlier crown, integration, signature, registry, or host-authority checks. +After those bindings are present, a successful smoke proves that the signed engine tree +routed through its intended seams for that session. It does not replace the earlier crown, +integration, signature, registry, or host-authority checks. ## Operator surface @@ -224,6 +290,8 @@ No live chain query, wallet, miner endpoint, evaluation incumbent, or referee da - [`stack_manifest.py`](https://github.com/latent-to/optima/blob/main/optima/stack_manifest.py) — integration records and release manifest - [`engine_tree.py`](https://github.com/latent-to/optima/blob/main/optima/engine_tree.py) — source promotion and deterministic tree materialization +- [`artifact_identity.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_identity.py) — reviewed direct-execution declaration identity +- [`cute_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/cute_cubin.py) — sealed CUBIN publication and runtime binding - [`model_provision.py`](https://github.com/latent-to/optima/blob/main/optima/model_provision.py) — model sealing - [`release.py`](https://github.com/latent-to/optima/blob/main/optima/release.py) — descriptor, artifacts, signing, publication, and context - [`release_host.py`](https://github.com/latent-to/optima/blob/main/optima/release_host.py) — registry reproducibility and host authorization diff --git a/docs/architecture/seam.md b/docs/architecture/seam.md index 7fd19b5..cea85ac 100644 --- a/docs/architecture/seam.md +++ b/docs/architecture/seam.md @@ -3,7 +3,8 @@ The seam connects Optima's stable slot ABI to version-pinned SGLang internals. It is intentionally split into two layers: - [`slots.py`](https://github.com/latent-to/optima/blob/main/optima/slots.py) defines the stable miner-facing semantic contract; -- `optima/seams.py` defines the SGLang-specific adapters that must be reviewed whenever the runtime pin moves. The current scheduler-gate row is on the post-main working branch pinned in [State of record](../reference/state-of-record.md). +- [`seams.py`](https://github.com/latent-to/optima/blob/main/optima/seams.py) defines the SGLang-specific adapter registry that must be reviewed whenever the runtime pin moves; +- [`seam.py`](https://github.com/latent-to/optima/blob/main/optima/seam.py) and the [`scheduler_gate` adapter](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_scheduler_gate.py) enforce activation and scheduler-only candidate loading. Slots are the narrow waist. Adapters are replaceable glue. @@ -42,7 +43,7 @@ flowchart LR Adding an adapter is one table change plus its implementation and tests. Bootstrap, activation, and compatibility derive their vocabulary from that table rather than maintaining parallel lists. -The current rows are: +The registered rows are: | Adapter | SGLang chokepoint | Served slot or role | Public binding | |---|---|---|---| @@ -60,7 +61,11 @@ The current rows are: Several adapters may share one binding when they implement one semantic product. The shallow AR-fusion consume adapter and both deep producer adapters share `arfusion`; activating only part of that set would violate the protocol. -The catalog can contain a verified slot before the pinned runtime exposes a safe live chokepoint. `attention.msa_block_score` currently has a slot and verifier contract, while [`sglang_msa.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_msa.py) deliberately refuses installation until a stable decode-side MSA chokepoint is registered. The prefill sibling has a live adapter and appears in the table. +The catalog can contain a verified slot before the pinned runtime exposes a safe live +chokepoint. `attention.msa_block_score` has a slot and verifier contract, while +[`sglang_msa.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_msa.py) +refuses installation unless a stable decode-side MSA chokepoint is registered. The +prefill sibling has an installed adapter and appears in the table. ## Bootstrap across spawned processes @@ -88,8 +93,8 @@ children stay pass-through and never emit an `active` receipt. SGLang versions that expose a plugin framework also have an [`sglang_plugin.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_plugin.py) shim. The `.pth` bootstrap remains the general spawn-safe path across the supported pin. Principal code: [`bootstrap.py`](https://github.com/latent-to/optima/blob/main/optima/bootstrap.py), -plus post-main working-tree paths `optima/seam.py` and -`optima/integrations/sglang_scheduler_gate.py` at the snapshot commit. +[`seam.py`](https://github.com/latent-to/optima/blob/main/optima/seam.py), and +[`sglang_scheduler_gate.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_scheduler_gate.py). ## Closed activation vocabulary @@ -242,10 +247,10 @@ The seam is deliberately pinned-runtime code. An SGLang upgrade requires: Optional backend rows declare `requires` packages so CPU/dev environments can skip an unassessable adapter rather than report a false break. Production arenas that depend on that adapter must provide and assess the dependency. -!!! warning "Current pin validation status" +!!! warning "Pin-validation boundary" A green import/chokepoint canary is necessary compatibility evidence, not proof that - the current pin preserves candidate execution and measured performance. Check - [State of record](../reference/state-of-record.md) for the dated claim boundary and + a pin preserves candidate execution and measured performance. Check + [State of record](../reference/state-of-record.md) for completed evidence and [SGLang compatibility](../dev/sglang-tracking.md) for the complete validation ladder. ### Diagnosing a seam failure diff --git a/docs/architecture/slot-contract.md b/docs/architecture/slot-contract.md index f1b960d..be4a61a 100644 --- a/docs/architecture/slot-contract.md +++ b/docs/architecture/slot-contract.md @@ -18,6 +18,12 @@ A slot may replace a bounded data-plane region, but it may not control final log For selection-style slots, the candidate fills an intermediate score sheet and the validator-owned tail performs the selection. This preserves the same property: the candidate never supplies the final selected output by assertion. +MSA keeps runtime selection width separate from verification policy. The live engine's +top-k controls the stock selection tail. `SlotSpec.correctness.top_k` only fixes the +width used when the verifier compares the highest-scoring blocks for overlap; equality +between those values is not a routing condition. The candidate always implements the +score-sheet boundary, not a miner-selected top-k operation. + ### 3. Correctness uses trusted high-precision ground truth Per-slot verification compares candidate output with a validator-owned fp32 or dequantized reference, never with the stock kernel as semantic truth. A faster implementation may use different reductions or low-precision arithmetic, so each slot selects an appropriate registered metric: @@ -170,6 +176,11 @@ A slot may expose several implementation variants for disjoint, validator-observ Eligibility is evaluated before candidate selection. Unknown capability fields, overlapping ambiguous variants, unsupported topology, and missing prerequisites fail closed or route to stock according to the registered pre-selection policy. The miner cannot introduce a new capability vocabulary through manifest extras. +Runtime-owned tuning phases are also outside candidate eligibility. While FlashInfer is +profiling autotuner tactics, both the deep MoE producer seam and its fused-epilogue +consumer use the stock path. Candidate code cannot affect tactic selection, and those +calls do not establish candidate firing evidence. + ## Atomic targets and composition A slot is a semantic ABI; a reward target is an economic identity. Most current targets are one-to-one singleton projections of slots, but the catalog can register an atomic target spanning multiple slots. diff --git a/docs/architecture/stacks.md b/docs/architecture/stacks.md index be626de..edb1ce7 100644 --- a/docs/architecture/stacks.md +++ b/docs/architecture/stacks.md @@ -59,7 +59,7 @@ These three identities answer different questions and should appear together in | Identity | Answers | Changes when | |---|---|---| | Stack manifest digest | Which semantic contributions and catalog are selected? | A target entry, runtime/base identity, arena context, or catalog snapshot changes | -| Engine-tree digest | Which exact deterministic materialized files were emitted? | Selected source closure, namespace rewriting, rebuild plan, or emitted inventory changes | +| Engine-tree digest | Which exact deterministic materialized files were emitted? | Selected source closure, direct-execution declaration, namespace rewriting, rebuild plan, or emitted inventory changes | | Launch identity | Which exact engine was executed? | Tree, native publication, model, image, topology, graph mode, workload, or role changes | Equal stack digests are not enough to claim equal execution if launch inputs differ. Equal @@ -179,6 +179,35 @@ the runtime result back to the prepared launch identity. Principal implementations are [`eval/engine_launch.py`](https://github.com/latent-to/optima/blob/main/optima/eval/engine_launch.py), [`eval/native_artifact.py`](https://github.com/latent-to/optima/blob/main/optima/eval/native_artifact.py), and [`eval/oci_backend.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_backend.py). +### Direct-artifact identity across the stack + +A direct artifact is not identified by its CUBIN name or `ops.entry`. Its +canonical execution identity covers the provider, compiler factory, allowed +profile inputs, ordered bindings, lifecycle plan, specialization predicates, +prelaunch operations, validator-owned resource plan, derived capability +requirements, and complete device launch plan. That projection is included in +the selected contribution and selected-delta identities used to materialize the +engine tree. + +The target catalog separately binds the immutable provider-registry snapshot and +digest. Launch preparation then adds the measured compile profile, native build +specification, sealed publication digest, and exact file inventory. Runtime +admission adds driver-observed CUBIN ABI and contract digests, while execution +receipts prove that the selected entry loaded, ran, and completed on every active +member without fallback. + +These layers answer different questions: + +| Layer | Bound authority | +|---|---| +| Contribution | Exact declarative device execution and source closure | +| Catalog | Which artifact providers and target features are permitted | +| Build | Image, logical/compiler architecture, topology-derived profile, patcher, and publication | +| Runtime | Exact retained CUBIN handle, observed ABI, rank device, parameters, resources, and lifecycle | +| Qualification | Full member coverage, successful invocation, seam completion, and zero fallback | + +See [Sealed direct artifacts](direct-artifacts.md) for the complete contract. + ## Transactional stack updates A passing qualification does not immediately mutate the incumbent. The settlement path requires two independently selected and reopened passing qualifications for the exact same reproduction identity. @@ -234,5 +263,7 @@ There is no supported arrow from a mutable miner URL, chain record, or evaluatio - [`stack_plan.py`](https://github.com/latent-to/optima/blob/main/optima/stack_plan.py) — marginal arms, cohorts, transitions, and rollback - [`engine_tree.py`](https://github.com/latent-to/optima/blob/main/optima/engine_tree.py) — deterministic source materialization and integration promotion - [`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py) — singleton, atomic, overlap, and composition policy +- [`artifact_identity.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_identity.py) — canonical direct-artifact execution identity +- [`artifact_provider.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_provider.py) — closed provider registry included in catalog identity - [`eval/reference_quality.py`](https://github.com/latent-to/optima/blob/main/optima/eval/reference_quality.py) — pristine reference quality products - [`eval/calibration.py`](https://github.com/latent-to/optima/blob/main/optima/eval/calibration.py) — calibrated qualification/reference policy diff --git a/docs/dev/gpu-setup.md b/docs/dev/gpu-setup.md index 22ec4e4..5ddd556 100644 --- a/docs/dev/gpu-setup.md +++ b/docs/dev/gpu-setup.md @@ -1,7 +1,7 @@ # GPU setup -This page prepares a contributor or maintainer GPU host for verification and -direct diagnostics. It is not a production arena deployment recipe. Production +This page prepares a contributor or maintainer GPU host for component verification and +performance development. It is not a production arena deployment recipe. Production candidate execution additionally requires the isolated worker and injected arena boundaries in the [validator guide](../validator-guide/first-hour.md). @@ -161,20 +161,20 @@ registered seam, completed it, and produced the expected output. A rank-local fallback would diverge the collective, so the candidate engine must abort rather than continue with mixed implementations. -## Direct model diagnostics +## Complete-engine performance development -`evaluate` and `bench` are development diagnostics. Use their live `--help` -and keep CUDA graphs enabled for performance work. Run them only on a disposable -host appropriate for candidate code: +Optima deliberately exposes no local qualification command. Contributors may profile +and A/B the complete serving engine on a disposable host appropriate for candidate code, +using the published arena contract as the environment specification. Keep the model, +runtime, topology, graph mode, workload, and charged-work basis fixed; measure the +candidate between two incumbent runs and distrust a delta smaller than baseline drift. +The canonical [performance-development procedure](../validator-guide/running-evals.md#performance-development) +defines the required inputs and local result record; no repository command materializes +this complete-engine bracket. -```bash -python -m optima.cli evaluate path/to/bundle \ - --model --tp-size -``` - -Production qualification uses complete B/C/B′ engines and pristine T through -an injected arena service; no direct command-line result substitutes for that -path. +Production qualification constructs complete B/C/B′ engines and a candidate-free +pristine T session through an injected arena service. A contributor-controlled model run +cannot substitute for that authority. ## From component proof to arena proof @@ -183,14 +183,13 @@ Move upward only after the lower layer is green: | Layer | Required observation | Still does not prove | |---|---|---| | Component `verify` | Registered reference and graph replay for exercised cases | Model integration or speedup | -| Direct `evaluate` / `bench` | Model can load and the seam can improve a diagnostic workload | Isolation, crown authority, independent reproduction | +| Local complete-engine A/B | Model can load and the selected delta can improve the matched workload | Validator isolation, hidden quality, crown authority, independent reproduction | | Arena screen | Static/build/ABI/graph/abbreviated-serving gates all promote | B/C/B′ drift, T quality, settlement | | Qualification PASS | Exact marginal complete-engine delta clears all registered gates | Crown until independent reproduction | -| Two-pass settlement | Reopened matching evidence creates a crown | Integration safety or release readiness | +| Two matching PASSes | Candidate is eligible for cohort settlement; the current registered cohort winner may be crowned while another valid pair is held | Integration safety or release readiness | -Keep direct diagnostic results because they are useful engineering evidence, -but label them as such. Do not copy their speedup into the current state of -record as if it came from the hardened qualification path. +Keep local A/B results as engineering evidence, labeled with their exact environment and +denominator. Do not treat them as qualification evidence. ## Common failures diff --git a/docs/dev/sglang-tracking.md b/docs/dev/sglang-tracking.md index f0ce80c..4bcedcf 100644 --- a/docs/dev/sglang-tracking.md +++ b/docs/dev/sglang-tracking.md @@ -2,14 +2,15 @@ Optima competes against and integrates with an exact SGLang runtime. The pin is part of evaluation and release identity, not a loose minimum version. The -current source pin is `0.5.13.post1` in +source pin is `0.5.13.post1` in [`optima/compat.py`](https://github.com/latent-to/optima/blob/main/optima/compat.py). -At the current snapshot, the static seam canary is green for `0.5.13.post1`, -while the GPU end-to-end broken-bundle rejection, faithful-bundle pass, and -champion rebaseline are still pending. `0.5.12.post1` remains the last pin with -complete retained H100 validation. Treat the source pin as the compatibility -target, not as proof that those empirical gates have already run. +A green static seam canary establishes import and chokepoint compatibility only. A +runtime pin is eligible for evaluation or release authority only after end-to-end GPU +controls reject a deliberately broken bundle, accept a faithful bundle, and rebaseline +the registered champions under the exact new identity. Treat the source pin as the +compatibility target, not as evidence that these empirical gates passed. Completed proof +coverage belongs in [State of record](../reference/state-of-record.md). ## Why the pin matters @@ -146,7 +147,7 @@ new one. Record whether each result is: - release build-and-serve evidence. This prevents a successful import check from being cited as a serving proof, -or an old H100 champion from being quoted as a current baseline. +or a champion from another runtime identity from being quoted as the active baseline. ## Release implications @@ -156,5 +157,5 @@ serving policy. Consumers verify those exact identities. Upgrading a running deployment means building and signing a new release; it is never an in-place package upgrade inside an existing release. -See [Current status](../reference/state-of-record.md) for which empirical GPU +See [State of record](../reference/state-of-record.md) for which empirical GPU proofs exist for the present architecture and which remain outstanding. diff --git a/docs/engine/integration.md b/docs/engine/integration.md index b5050eb..eedcafa 100644 --- a/docs/engine/integration.md +++ b/docs/engine/integration.md @@ -6,7 +6,7 @@ an automatic consequence of settlement. The integration reviewer is not re-running the economic contest or granting a crown. They are answering a narrower product question: can the exact selected delta that won be -owned, understood, maintained, composed with today's reviewed Engine, and distributed +owned, understood, maintained, composed with the reviewed release baseline, and distributed under an explicit provenance and security record? ## Inputs diff --git a/docs/engine/overview.md b/docs/engine/overview.md index 76fee18..3e17211 100644 --- a/docs/engine/overview.md +++ b/docs/engine/overview.md @@ -66,12 +66,37 @@ rather than trusting modes, filenames, or build logs. | SBOM and provenance | Which declared inputs and build lineage produced the release artifacts? | | `ServeSpec` | Which image, platform, topology, command, environment, and mount contract may run? | | Signed descriptor | Who authorized this exact closed set of identities? | -| Registry reproducibility statement | Did two controlled builds publish the same raw image digest? | -| Serve receipts | Did every expected slot and tensor-parallel rank activate, fire, and complete in the smoke? | +| Registry reproducibility statement | Did two reopened registry manifests expose the same raw image digest and required labels? | +| Serve receipts | Did the supplied receipt set report active, fired, and completed coverage for every expected slot and tensor-parallel rank? | These products compose; they are not interchangeable. For example, a valid signature over a descriptor with a missing model tree still cannot start, and a responsive server with fallback receipts has not proved that the released optimization executed. +The production authority must additionally bind both registry readbacks to their +controlled builder invocations, bind smoke receipts to the exact release and serving +session, and require direct-AOT load/invocation evidence when the release declares it. +Those causal bindings are not established by the registry or receipt primitives alone. + +## Native and dependency artifacts + +Architecture coverage is explicit rather than inferred from the machine that performed +the build. The FlashInfer dependency-patch path publishes one sealed overlay containing +the validator-declared `sm100` and `sm103` fused-MoE modules. FlashInfer captures its CUDA +architecture list in import-time state, so each module is compiled in a separate hermetic +child interpreter whose environment is fixed before any FlashInfer import; the parent +build process never imports FlashInfer. The build child compiles but does not load the +module. At engine startup, the installer selects only the row matching the current device +and refuses an architecture that the publication does not cover. + +This multi-architecture dependency overlay is distinct from a sealed direct artifact: it +patches a closed dependency subtree and publishes exact prebuilt dependency modules, +while a direct artifact is admitted through its registered provider, slot ABI, compile +profile, resource plan, and launch plan. + +FlashInfer autotuning is a runtime-owned phase. While its tactic profiler is active, both +the deep MoE producer and the fused-epilogue consumer stay on the stock path. A reviewed +or candidate implementation cannot influence tactic selection, and autotuner calls do +not count as proof that an Optima seam fired. ## Serving boundary diff --git a/docs/engine/release-workflow.md b/docs/engine/release-workflow.md index ca25011..37ba14e 100644 --- a/docs/engine/release-workflow.md +++ b/docs/engine/release-workflow.md @@ -5,6 +5,15 @@ verifies releases and emits container contexts; it intentionally has no `release-create` command that could collapse review, key access, and publication into an ambient shell operation. +!!! danger "Release acceptance is conjunctive" + A deployable release requires every gate on this page: clean installation and runtime + import, signed native-profile propagation, closed effective SGLang arguments and + management routes, builder-output-to-registry binding, externally anchored signatures, + exact host policy, and release/session-bound execution receipts. A missing gate is a + refusal, not a warning that another artifact can override. The dated + [State of record](../reference/state-of-record.md) reports which gates the pinned + implementation and retained evidence actually satisfy. + This is deliberate separation of duties. The builder may assemble and hash inputs without possessing the private signing key. The signer may approve one descriptor without gaining permission to mutate its artifacts. The publisher may stage a signed release without @@ -37,8 +46,10 @@ Freeze and independently reopen: - a `ServeSpec` with digest-pinned base image, platform, model mount, TP size, canonical arguments, and bounded environment. -The serving specification cannot override validator-owned model or TP flags and -rejects dangerous environment keys. +Release authorization must prove that the effective SGLang argument vector cannot +override validator-owned model, TP, loader, network, or management policy through exact +flags, aliases, abbreviations, duplicates, or configuration indirection. Environment +keys require the same closed treatment. Exact-name filtering alone is insufficient. ## Construction sequence @@ -165,23 +176,38 @@ model/native identities, upstream identities, and serving specification. ## Publication checklist -!!! note "Implemented workflow, pending final operational proof" - The typed release, registry, authorization, runtime, and receipt gates are - implemented and covered by focused/full-suite tests. The current state of - record does not claim an actual reproducible registry image pair followed - by final TP8 serving and receipt verification. +!!! note "Operational proof requirement" + Unit and slice coverage of the typed release, registry, authorization, runtime, and + receipt gates does not authorize production rollout. Release evidence must include + clean installation from the emitted wheel, provider-specific native reopening, + authenticated builder outputs, closed effective serving policy, a reproducible + registry image pair, and descriptor/session-bound serving evidence at the approved + topology. Completed evidence is tracked in + [State of record](../reference/state-of-record.md). - Rebuild from a clean, exact source revision. -- Require the current SGLang pin and compatible upstream revision. +- Require the declared SGLang pin and compatible upstream revision. - Verify every integrated source digest and exact review-record coverage; retained external review artifacts were authenticated during promotion. - Use a digest-pinned base image; tags are not release identities. - Confirm source and wheel reproducibility before signing. +- Install the emitted wheel in a clean environment and import/exercise every runtime + entrypoint reachable from the signed manifest; source-tree imports are not this proof. +- For each sealed direct artifact, retain the provider-specific prebuild, compile-profile, + index, observed-ABI, and publication reopen products before converting them into the + generic native release identity. +- Prove that effective SGLang arguments, environment, loader, network, and management + routes equal the signed policy after parsing—not merely that a short denylist is absent. +- Bind each no-cache builder's emitted output to the registry object admitted into the + reproducibility statement. - Keep the private key offline from the candidate and serving contexts. - Verify the published tree using the externally distributed public key and expected descriptor digest. - Record the descriptor digest as the release identifier; never use a mutable `latest` path as authority. +- Bind every serve receipt to the release descriptor and one smoke-session identity. For + sealed direct artifacts, require per-rank AOT load and invocation coverage in addition + to ordinary seam coverage. ## Failure behavior @@ -189,12 +215,15 @@ model/native identities, upstream identities, and serving specification. |---|---| | Missing or extra integration review | Reject release preparation; do not drop the unmatched target | | First and second source/wheel products differ | Stop and diagnose nondeterminism before signing | +| Emitted wheel fails a clean import or entrypoint exercise | Reject the release; repair the complete runtime-module allowlist and rebuild | | Native publication or model receipt no longer reopens | Stop; recover the exact input or construct a new release identity | +| Generic native inventory exists without provider-specific profile/index/ABI proof | Reject provider-specific release admission | | Signature verifies only with the key embedded in the release | Untrusted until an externally obtained expected key verifies it | | Destination digest directory exists but differs | Treat as corruption/collision; never overwrite in place | +| Builder output is not cryptographically bound to the registry object being signed | Do not issue reproducibility authorization; tag readback alone is insufficient | | Two container builds produce different raw manifests | Do not issue reproducibility authorization | | Image verifies but host policy differs from signed `ServeSpec` | Do not start the container | -| Canary answers but receipts show fallback/missing rank coverage | Do not promote; responsiveness is not activation proof | +| Canary answers but receipts are unbound, omit AOT evidence, show fallback, or miss rank coverage | Do not promote; responsiveness is not activation proof | An aborted release attempt leaves the previous signed publication authoritative. Repairing inputs should create a newly prepared and signed descriptor, not a hand-edited release diff --git a/docs/engine/verify-run.md b/docs/engine/verify-run.md index 2fe0bc8..06a0166 100644 --- a/docs/engine/verify-run.md +++ b/docs/engine/verify-run.md @@ -21,6 +21,15 @@ externally trusted key + expected descriptor Each step authenticates a different boundary. Skipping from a valid release directory to `docker run ` drops the registry, host-policy, and activation proofs. +!!! danger "Verification primitives are not rollout authority" + The commands and APIs below authenticate individual boundaries. Rollout additionally + requires clean runtime-wheel import, signed native-profile propagation, closed + effective SGLang arguments and management routes, authenticated builder-output + binding, and release/session-bound execution receipts. If any product is absent, stop + before authorization or start. The dated + [State of record](../reference/state-of-record.md) records the pinned implementation's + completed and blocked gates. + ## Reopen and verify ```bash @@ -69,13 +78,16 @@ and emits a frozen deterministic context containing: No chain endpoint, wallet, proposal URL, or private signing material enters the context. -## Build and deploy +## Implemented host primitives -The closed host workflow is programmatic: +The host module exposes these programmatic primitives: 1. `publish_container_twice()` reopens the context, runs two no-cache BuildKit builds, pushes them under distinct temporary tags, reopens both Registry-v2 - manifests, requires one raw digest, and signs that common identity. + manifests, requires one raw digest, and signs that common registry identity. A + production implementation must additionally authenticate that each readback is the + result emitted by its corresponding builder invocation. The function's current return + value does not supply that builder-output binding. 2. `authorize_release_container()` binds the signed statement and reopened registry image to independently supplied public-key, release-descriptor, platform, repository, and image-digest expectations. @@ -87,9 +99,12 @@ The closed host workflow is programmatic: directory for `verify_serve_receipts()`; `destroy()` force-removes the container and volumes. -A compact handoff looks like this. Values prefixed with `approved_` must come -from the deployment's independent trust and rollout records; do not derive -them from the registry object you are trying to authorize. +The following composition is useful for integration testing of those primitives. It is +not a production authorization recipe: before executing it on a release path, deployment +orchestration must add the missing builder-output, clean-wheel, effective-argument, +management-route, native-profile, and receipt-session gates. Values prefixed with +`approved_` must come from independent trust and rollout records; do not derive them from +the registry object being authorized. ```python from optima.release import verify_serve_receipts @@ -100,7 +115,8 @@ from optima.release_host import ( publish_container_twice, ) -# Build twice, reopen both Registry-v2 manifests, require one digest, and sign it. +# Integration composition only. The caller must separately authenticate that each +# registry readback came from its corresponding builder output. signed_build = publish_container_twice( context_root, repository=repository, @@ -140,29 +156,39 @@ finally: handle.destroy() ``` -The code intentionally leaves the smoke function and rollout approval step to -deployment orchestration. Neither is an authority that `release_host.py` can -manufacture on its own. If any operation before `try` fails after container -creation, `create_release_container()` performs best-effort cleanup; after it -returns a handle, the caller owns the guaranteed `destroy()` path. +The code intentionally leaves every missing gate, the smoke function, and rollout +approval to deployment orchestration. `release_host.py` cannot manufacture those +authorities. In particular, the shown `receipt_proof` checks coverage in a directory; it +does not bind the receipt frames to this descriptor/session or check direct-artifact AOT +coverage. If any operation before `try` fails after container creation, +`create_release_container()` performs best-effort cleanup; after it returns a handle, the +caller owns the guaranteed `destroy()` path. -The current host contract requires a read-only root, all capabilities dropped, +The host contract requires a read-only root, all capabilities dropped, `no-new-privileges`, reviewed seccomp, a read-only model bind, a 16 GiB `/tmp` tmpfs (`nosuid,nodev,noexec`), 32 GiB shared memory, host IPC, host networking, and the exact signed entrypoint/arguments/environment. It requests all GPUs visible to that host boundary and verifies Docker's `Count=-1`; outer scheduling must expose only the intended device pool. +Host networking also exposes the server's management surface. Release authorization must +disable or authenticate every management route; successful image/argument inspection +alone does not provide that control. + Unlike the candidate referee, the serving container is intentionally not -no-egress. The current host verifier also does not enforce `Config.User` or add +no-egress. The host verifier also does not enforce `Config.User` or add a Dockerfile `USER`; base-image and release review must own that decision. Do not document the serving boundary as non-root until code enforces it. -## Startup and rollback +## Production startup and rollback contract + +Perform this sequence only after the [release acceptance checklist](release-workflow.md#publication-checklist) +and the dated [State of record](../reference/state-of-record.md) show that every required +gate is implemented and retained: 1. Verify the release and context from immutable storage. -2. Produce and authenticate the reproducible registry identity through the - closed host APIs above. +2. Produce the reproducible registry identity through the closed host APIs and + authenticate each registry object against its corresponding builder output. 3. Confirm model content and image digests before scheduling. 4. Create, inspect, and start a canary with the signed arguments and exact host policy. @@ -170,17 +196,24 @@ not document the serving boundary as non-root until code enforces it. 6. Copy the fresh receipt directory and run `verify_serve_receipts()` against the reopened release. It requires `active`, `fired`, and `completed` coverage for every released slot and TP - rank, with no `load_failed` or `fallback` receipt. -7. Promote by immutable descriptor/image identity and retain the resulting - `ServeReceiptVerification` digest. + rank, with no `load_failed` or `fallback` receipt. Independently require the input + frames to bind this release descriptor and serve-session identity. For a release with + sealed direct artifacts, require per-rank `aot_loaded` and `aot_invoked` coverage as + well; the current generic verifier does not enforce either additional condition. +7. Promote by immutable descriptor/image identity only after every external and typed + gate succeeds. Retain the bound receipt product; an unbound + `ServeReceiptVerification` digest is not promotion authority. 8. Destroy the canary; roll back by selecting a previously verified release, never by mutating the current tree in place. -!!! note "Operational status" - These gates are implemented and tested, but the current evidence ledger - does not claim the final real registry double-build, TP8 serve, and receipt - smoke have been completed. +!!! note "Operational proof requirement" + Unit and slice tests do not authorize rollout. The release must retain a clean-wheel + import/entrypoint proof, provider-specific native authority, authenticated builder + outputs, closed serving arguments and management routes, a serve smoke at the approved + topology, and descriptor/session-bound receipts (including AOT coverage when + applicable). See + [State of record](../reference/state-of-record.md) for completed evidence. Serving does not require chain access. A chain outage can delay new proposal evaluation or attribution updates, but it cannot prevent rebuilding or running diff --git a/docs/get-started/concepts.md b/docs/get-started/concepts.md index 635f08e..ad20ffa 100644 --- a/docs/get-started/concepts.md +++ b/docs/get-started/concepts.md @@ -154,12 +154,12 @@ oracle and prevents chain state from leaking into deployment. ## Authority levels -Optima exposes useful developer commands that do not create crowns: +Optima exposes contributor diagnostics that do not create crowns: - `scan` checks static policy without loading a bundle; -- `verify` compares a slot implementation with its trusted reference; -- `evaluate` measures direct model behavior and throughput; and -- `bench` adds task diagnostics. +- `verify` compares a slot implementation with its trusted reference; and +- contributor-controlled matched A/B profiling can investigate full-engine behavior and + throughput without producing validator evidence. Production authority begins only after finalized intake, immutable publication, a closed arena service, isolated qualification, retained evidence, and independent reproduction. diff --git a/docs/get-started/quickstart.md b/docs/get-started/quickstart.md index 2c89252..d0280d7 100644 --- a/docs/get-started/quickstart.md +++ b/docs/get-started/quickstart.md @@ -44,7 +44,7 @@ python -m optima.cli scan examples/miner_silu_torch `scan` parses the manifest and applies the recursive static policy. A clean scan is an admission signal, not a sandbox and not correctness evidence. -A clean example currently looks like this: +Expected output for the clean example: ```text bundle: example-silu-torch-cpu abi: optima-op-abi-v0 ops: 1 @@ -135,8 +135,8 @@ The quickstart did not exercise: - release construction and serving. Those boundaries require validator-owned hardware, runtime identities, policies, and -evidence stores. Direct `evaluate` and `bench` commands are useful GPU development -diagnostics, but they remain non-authoritative. +evidence stores. Developer GPU experiments remain non-authoritative regardless of their +local output. ## Continue diff --git a/docs/history/changelog.md b/docs/history/changelog.md index 2a44e4c..2d6dc9f 100644 --- a/docs/history/changelog.md +++ b/docs/history/changelog.md @@ -262,10 +262,10 @@ The exact final structural proof used a synthetic 100-ppm candidate to exercise Changes: [PR #55](https://github.com/latent-to/optima/pull/55) and [PR #56](https://github.com/latent-to/optima/pull/56). The latter only clarified that the value embedded in the container context is an Ed25519 public verification key. -## 2026-07-13 — B300 intake findings after main +## 2026-07-13 — CuTe admission and scheduler-role loading -Three focused changes landed on the `fix/b300-intake-findings` working branch after -the PR #56 main snapshot: +The initial B300 intake work added three boundaries that were later included in +PR #59: - `804147c4` admitted CuTe's function-object `cute.compile(...)` idiom under conservative @@ -281,10 +281,60 @@ the PR #56 main snapshot: refuses a string/bytes literal as the first positional argument; CuTe remains the sole admitted entry and Triton requires no explicit `.compile` row. -These are current working-tree contracts, not merged pull requests or commits -reachable from an `origin/*` ref at this snapshot. Focused regressions and the -current 1,724-passed/18-skipped/0-failed suite are green; the final B300 joined -qualification remains a separate evidence claim. +These changes were prerequisites for the later sealed direct-artifact path; they did not +by themselves make runtime CuTe host objects an acceptable crown surface. + +## 2026-07-13 to 2026-07-15 — Sealed direct artifacts + +The artifact boundary gained a closed provider registry and the authoritative +`cutlass.cute.cubin.v1` provider. Candidate factory code executes in a GPU-hidden, +no-network prebuild child. The retained product is an exact-architecture CUDA ELF CUBIN +plus validator-reconstructed call, launch, resource, specialization, and lifecycle +authority. + +The runtime binds complete physical kernel inventories to logical exports by ordinal +after rank-local CUDA and group initialization. Validator-owned materialization covers +pointer/scalar parameters, packed fields, CUDA 13 TMA descriptors, distinct CUTLASS and +CuTe FastDivmod representations, launch geometry, optional clusters, streams, generated +resources, and teardown. PTX, host objects, shared libraries, candidate launch callbacks, +and runtime JIT inputs are outside the provider contract. + +Ledger-attested B300 TP4 blockscore runs reached complete 4/4 load, invocation, and +completion with zero fallback; the pod-local receipts were not retained. The subsequent +TP4 B200 joined qualification retained primary and reproduction PASSes at 1.0561× and +1.0487× on the charged basis. Settlement used 1.0487×, committed a generation-1 crown, +and reopened it after restart. The run proves the blockscore direct-CUBIN lane, not +arbitrary direct-artifact collectives. + +Changes: [PR #59](https://github.com/latent-to/optima/pull/59), including +`368727f6` through `93dfb182` and their validation follow-ups. + +## 2026-07-15 — Chain reconciliation and executable-surface trim + +Finalized-height refresh now precedes settlement leasing and commit. Raw reveal history +decoding is bounded and requires event/storage equality. Failed extrinsics report +`submitted=false`, and a retained pending weight intent can be reopened across later +heads without resubmission. + +The superseded local JSON commit/reveal simulator and direct evaluator/benchmark surface +were removed. The public local contribution loop is `scan` plus `verify`; complete-engine +speed and quality remain registered-arena authorities. + +Changes: `5afdf2e2`, `1796fc24`, `a52712ef`, `e9ac2f30`, `030d619c`, and +`28d8ff29`; the trim reached `main` through PR #58. + +## 2026-07-16 — Shared strict validation and core wheel smoke + +The `refactor/strict-validation-kernel` branch centralizes canonical digest, integer, +identifier, exact-field, duplicate-key, CUDA-driver-integer, and environment-flag checks +in `optima/_strict.py`. Calling modules retain their own exception types, grammars, and +bounds. Most authority digests now reject the all-zero placeholder; digest computation is +unchanged. + +A follow-up adds `_strict.py` to the serving-wheel allowlist and exercises selected core +imports from a clean extracted wheel. The broader manifest/direct-artifact runtime closure +is still absent, so the release gate must expand from that core smoke to every +manifest-reachable serving entrypoint before the branch is release-ready. ## Corrections and retractions @@ -319,14 +369,22 @@ The live slot catalog contains 11 slots, not 10. `attention.msa_prefill_block_sc | Transactional settlement and emissions | #54 | `084fb8d7` | | Arena, independent reproduction, integration, release | #55 | `b218190d` | | Public-key CodeQL clarification | #56 | `990721bc` | -| Post-main CuTe admission, scheduler-only load gate, and shared DSL-JIT policy | Working branch | `804147c4`, `3cff4731`, `5325468a` | +| Direct CUBIN provider, B300/B200 intake fixes, chain reconciliation | #59 | `93fe837b` | +| Legacy simulator and evaluator removal | #58 (including the stacked trim) | `6d7e92ef` | +| Shared strict validation and core wheel-import regression | Branch-only | `4c3eba59`, `7c171d30`, `f8be20a4` | ## Current interpretation -The history resolves into three layers: +The history resolves into four layers: 1. early mechanism work established slots, seams, quality gates, collectives, and live chain transport; 2. historical B300 runs demonstrated that shallow, deep, and MSA-prefill optimizations could produce real speedups; -3. the July hardening rebuilt authority so those ideas can be evaluated marginally, reproduced independently, settled transactionally, integrated as reviewed source, and shipped without chain dependencies. - -Only the third layer defines current crown and release authority. See [State of record](../reference/state-of-record.md) for the remaining validation debt and [Architecture overview](../architecture/overview.md) for the stable model. +3. the July hardening rebuilt authority so those ideas can be evaluated marginally, + reproduced independently, and settled transactionally; and +4. the direct-artifact provider exercised that authority for blockscore while preserving a + separate integration and release gate. + +Only retained current-schema qualification defines crown authority, and a crown still +does not define release authority. See [State of record](../reference/state-of-record.md) +for the remaining validation debt and [Architecture overview](../architecture/overview.md) +for the stable model. diff --git a/docs/index.md b/docs/index.md index b8591d9..aaa7225 100644 --- a/docs/index.md +++ b/docs/index.md @@ -83,15 +83,16 @@ copying them. | Deploy intake, an arena provider, and qualification workers | [Validator guide](validator-guide/overview.md) | | Verify or operate a signed serving artifact | [Optima Engine](engine/overview.md) | | Audit trust boundaries and failure behavior | [Security model](security/threat-model.md) | -| Check what is implemented, measured, and still unproven | [Current status](reference/state-of-record.md) | +| Check what is implemented, measured, and still unproven | [State of record](reference/state-of-record.md) | ## Evidence is scoped, not blended -Optima has historical B300 performance wins and a completed hardened authority -implementation. Those are different evidence classes. The historical wins were measured -through the earlier referee; the hardened path has structural RTX evidence but still -requires current-path B300 qualification and a real release-image build-and-serve proof. -The [state of record](reference/state-of-record.md) keeps those claims separate. +Every performance or authority claim is scoped to the exact runtime, hardware, arena, +stack, identities, and procedure that produced it. Diagnostic measurements cannot +authorize a crown; crown evidence cannot authorize reviewed source; and release +verification cannot retroactively validate qualification. The +[state of record](reference/state-of-record.md) identifies which evidence products have +actually been retained for each boundary. !!! note "Source of executable truth" The [Optima source repository](https://github.com/latent-to/optima) owns executable diff --git a/docs/miner-guide/bundle-format.md b/docs/miner-guide/bundle-format.md index 7bbc99a..bb67c1e 100644 --- a/docs/miner-guide/bundle-format.md +++ b/docs/miner-guide/bundle-format.md @@ -25,9 +25,10 @@ my_bundle/ ``` Advanced, target-approved bundles may also contain declared `.cu`/`.cuh` -sources, unified dependency diffs, and a reviewed `rebuild.json`. Do not include -compiled objects, wheels, shared libraries, caches, model weights, credentials, -or absolute machine paths. +sources, unified dependency diffs, a reviewed `rebuild.json`, or a CuTe source +module with a sealed direct-artifact declaration. Do not include compiled +objects, CUBINs, wheels, shared libraries, caches, model weights, credentials, or +absolute machine paths. All declared paths are bundle-relative. Intake rejects path traversal, unsafe filesystem entries, and undeclared executable material. The fetched archive is @@ -153,6 +154,8 @@ Each `[[ops]]` row describes one implementation: | `setup` | engine-wide setup hook; currently forbidden by every registered target | | `base_kernel`, `override_point` | registered override composition fields | | `cuda_sources` | declared inspectable `.cu`/`.cuh` inputs to an approved build step | +| `aot_exports` | sealed compiler exports, semantic bindings, specializations, lifecycle steps, and complete device plans | +| `artifact_resources` | miner-named but validator-allocated workspace, prepared storage, or engine state | `bundle_id` must be a simple non-empty identifier, and the component ABI is currently exactly `optima-op-abi-v0`. @@ -239,6 +242,81 @@ convenience. If two rows match the same call, the validator cannot know which se delta was measured or which implementation would later serve. Conversely, a gap is allowed: the trusted incumbent remains the fallback outside all candidate domains. +## Direct CUBIN bundles + +The registered direct-artifact provider is `cutlass.cute.cubin.v1`. It compiles +CuTe source into a sealed CUBIN during validator prebuild and executes it through +the validator's CUDA Driver ABI. It does not run a miner-supplied launcher in the +scheduler process. + +A direct bundle contains source and declarations, not generated native output: + +```text +my_direct_bundle/ + manifest.toml + kernels/ + kernel_cute.py + metadata/ + kernel.json + rebuild.json + README.md + LICENSE +``` + +The resolved rebuild selection is data, not a command: + +```json +{ + "steps": [ + {"type": "repo_python", "path": "build_cute_cubin.py"} + ] +} +``` + +The operation row retains `entry` because the outer bundle ABI requires it. For +direct execution the value is not called and is excluded from canonical direct- +execution identity. The meaningful declaration is `[[ops.aot_exports]]`, which +must include the provider, compiler-side factory, bounded compile-profile inputs, +ordered bindings, specialization/lifecycle fields, and a complete +`optima.device-launch-plan.v1` plan. + +The plan is intentionally explicit. It inventories every logical kernel and +formal parameter width, then declares every launch's grid, block, optional +cluster, shared memory, parameter construction, stream, and allowlisted +attributes. Parameter values are constructed by validator code from checked +expressions and admitted slot resources. Supported constructions include exact +scalars and pointers, packed structs, TMA descriptors, CuTe/CUTLASS FastDivmod, +and provider-authorized group projections. + +If the algorithm needs intermediate storage, declare +`[[ops.artifact_resources]]`. `workspace.*` is call-local, `prepared.*` crosses +the validator's prepare/run boundary, and `state.*` persists with the engine +entry. The validator allocates all three. Resource declarations cannot create a +new stream, group, semantic input, output, or arbitrary address. + +Direct-artifact admission is exact: + +- the target must permit `aot:cutlass.cute.cubin.v1` and + `rebuild:build_cute_cubin`; +- the factory runs only in the no-network/no-GPU compiler child with declared + validator-measured profile values; +- the only executable products are CUBINs; the sealed provider publication also carries + its canonical index, while the enclosing native publication carries its prebuild and + inventory metadata, all bound to build/profile/tree identities; +- the scheduler admits the complete driver-observed CUBIN ABI by kernel ordinal + after rank CUDA setup; and +- qualification requires load, invocation, and seam-completion evidence on every + active member, with no fallback. + +The provider vocabulary includes native group handles and peer-pointer tables, but +the standard `build_cute_cubin` load path does not install the required group +capability/handle resolvers; such a plan fails closed before execution. Group-aware +support therefore requires reviewed runtime work before the ordinary distributed +correctness, graph, full-engine, and reproduction gates can produce evidence. + +Use the field tables in [Bundle manifest](../reference/manifest-schema.md) and the +runtime model in [Sealed direct artifacts](../architecture/direct-artifacts.md). + ## What the content hash binds `chain-package` computes proposal identity from every regular identity-bearing file in @@ -262,6 +340,13 @@ The archive URL is transport, while the content hash is proposal identity. The v may retry transient transport for the same hash, but it must never accept different extracted bytes under that identity. +For a direct row, a second canonical identity covers the executable declaration: +provider, factory, profile inputs, bindings, resources, lifecycle, specialization, +prelaunch, and the complete device plan. Copy/provenance analysis combines that +declaration with the normalized transitive Python source closure. Reformatting or +renaming an unused `entry` does not change direct execution, while any +execution-bearing declaration change does. + ## Prepare/forward slots MoE slots use a load-time preparation callable plus a serving callable: @@ -334,5 +419,9 @@ override, CUDA-source, and dependency-patch features and resolves them against the target catalog. See [Override points](override-points.md) and [Dependency patches](dep-patches.md). +Direct-artifact exports follow the same rule: the closed provider registry and +target catalog, not a provider string in TOML, determine whether a row is +authoritative and crownable. + Next, read the [Kernel ABI](kernel-abi.md), then build the minimal example in [Your first kernel](your-first-kernel.md). diff --git a/docs/miner-guide/dep-patches.md b/docs/miner-guide/dep-patches.md index 4d5bc34..89e18a1 100644 --- a/docs/miner-guide/dep-patches.md +++ b/docs/miner-guide/dep-patches.md @@ -7,17 +7,16 @@ depends on a small producer/export change in that dependency. It is not a framework-mode escape hatch, a way to edit `site-packages`, or a way to execute a bundle-supplied installer. -## Current policy +## Registered dependency-patch policy -The only patchable dependency is currently `flashinfer`. The policy permits -files matching: +The dependency-patch allowlist admits only `flashinfer` files matching: ```text flashinfer/data/csrc/fused_moe/* ``` -The target catalog currently adds the FlashInfer patch features only to the -deep `collective.moe_finalize_ar_rmsnorm` singleton target and the +The target catalog grants the FlashInfer patch features only to the deep +`collective.moe_finalize_ar_rmsnorm` singleton target and the `collective.moe_epilogue.v1` atomic target. A patch on another registered target does not gain permission merely because the manifest parses. @@ -80,7 +79,9 @@ approved extension builder too: } ``` -Those are the only registered rebuild patchers. The parser rejects +Those are the registered dependency-patch rebuild patchers. The same closed registry also +contains `build_cute_cubin.py` (`optima.build-cute-cubin.v1`) for the sealed +direct-artifact provider; it is not a dependency-patch installer. The parser rejects `bundle_python`, arbitrary repository scripts, duplicate patchers, unknown fields, and unregistered step types. The validator snapshots each patcher's ID and source digest; the bundle chooses a reviewed capability, not executable @@ -147,7 +148,7 @@ The committed [atomic stack fixture](https://github.com/latent-to/optima/tree/main/tests/fixtures/stack_fused_epilogue_atomic) demonstrates manifest identity, patch declaration, and rebuild selection. Its native and Python bodies are deliberately test stubs and do not implement the -current deep ABI. Treat it as a schema fixture, not a miner kernel template. +production deep ABI. Treat it as a schema fixture, not a miner kernel template. If the desired edit is outside this closed policy, submit it through the [Discovery lane](discovery-lane.md) rather than disguising it as a dependency diff --git a/docs/miner-guide/diagnostics.md b/docs/miner-guide/diagnostics.md index c7809ed..0c8fb1b 100644 --- a/docs/miner-guide/diagnostics.md +++ b/docs/miner-guide/diagnostics.md @@ -182,8 +182,8 @@ Common causes are host synchronization, data-dependent Python branching, capture-time compilation/allocation, stale pointers, partial replay writes, or collective ordering changes. `graph_safe: true` is only a declaration. -Do not “fix” graph failure by using `evaluate --disable-cuda-graph`; that changes -the serving regime. See [Graph evidence](graph-safety.md). +Do not “fix” graph failure by disabling CUDA graphs in a local profile; that changes the +serving regime. See [Graph evidence](graph-safety.md). ## 5. Chain payload, fetch, and publication @@ -248,12 +248,13 @@ Quality problems: - required fidelity/task metric regresses: candidate FAIL; - stock/reference drift overlaps the calibrated boundary: `NO_DECISION`; -- pristine T or raw evidence cannot reopen: authority/infrastructure failure; -- a local `bench` pass differs: local benchmark flags and prompts are not the +- referenced pristine-T identity or raw quality artifact cannot reopen: + authority/infrastructure failure; +- a contributor-controlled quality check differs: local flags and prompts are not the bound reference policy. -`evaluate` and `bench` explicitly print diagnostic, non-authoritative results. -They are for reproducing mechanisms, not contesting a retained validator grade. +Contributor-controlled matched A/B profiling can reproduce a mechanism, but it cannot +contest a retained validator grade or create qualification authority. ## 8. Reproduction diff --git a/docs/miner-guide/discovery-lane.md b/docs/miner-guide/discovery-lane.md index 1b7d0e3..40ef757 100644 --- a/docs/miner-guide/discovery-lane.md +++ b/docs/miner-guide/discovery-lane.md @@ -15,7 +15,7 @@ instead. | Question | Component proposal | Discovery proposal | |---|---|---| | What is being replaced? | one registered singleton or complete registered atomic target | an exact bounded patch that does not fit a current target | -| ABI | `optima-op-abi-v0` callable rows | `optima-discovery-abi-v1` closed patch manifest | +| ABI | `optima-op-abi-v0` registered slot rows, including sealed direct artifacts | `optima-discovery-abi-v1` closed patch manifest | | Source shape | manifest, declared entry/metadata, and allowed advanced inputs | exactly the manifest and declared unified diffs | | Runtime authority | validator dispatches through a registered slot seam | validator applies a sealed overlay through fixed discovery process roles | | Economic result | standing target crown may participate in emissions | one bounded, non-renewable, expiring bounty claim when enabled | @@ -36,6 +36,14 @@ outside every slot. Conversely, a CUDA implementation is not automatically disco declared `.cu` source can belong to a component target that allows its registered rebuild feature. +The same rule applies to native publication paths. A `cutlass.cute.cubin.v1` export remains +a component proposal when it implements an existing registered target and that target +allows the provider's manifest and rebuild features. A policy-constrained FlashInfer +dependency patch can also remain a component advanced input. Its sealed overlay carries +the validator-declared `sm100` and `sm103` modules, builds each architecture in a separate +hermetic child before any FlashInfer import, and installs only the row matching the live +device. Neither mechanism grants authority to patch arbitrary engine control flow. + Use these examples: - a faster algorithm behind the exact `attention.sdpa` signature: component singleton; @@ -56,8 +64,8 @@ makes overlap, fallback, attribution, and standing rewards well-defined. ## Discovery is a separate ABI -A discovery proposal is **not** an op bundle and must not use legacy -`[competition] mode = "system"`. Its closed manifest uses +A discovery proposal is **not** an op bundle, and +`[competition] mode = "system"` is not a valid discovery schema. Its closed manifest uses `optima-discovery-abi-v1` and contains only exact text patches plus applicability and build-profile claims. @@ -152,6 +160,10 @@ tree. It then: 5. activates that sealed overlay only through the validator's fixed process-role machinery inside the isolated engine. +This overlay is a frozen patched SGLang source tree. It is not the FlashInfer dependency +overlay or a direct CUBIN publication; those component mechanisms retain their own closed +build, architecture, ABI, and runtime-selection policies. + The discovery C arm is ephemeral and bound to the incumbent stack/tree, proposal digest, policy digest, build-profile digest, and overlay identity. It does not add a standing entry to the incumbent evaluation stack. diff --git a/docs/miner-guide/examples.md b/docs/miner-guide/examples.md index 64521e6..21af308 100644 --- a/docs/miner-guide/examples.md +++ b/docs/miner-guide/examples.md @@ -4,9 +4,9 @@ The repository examples are development fixtures for specific parts of the component ABI. They are not a catalog of active crowns, and their comments or metadata are not validator decisions. -Several examples predate explicit contribution targets and rely on legacy -singleton resolution. When starting a real proposal, add the appropriate -`[competition]` table and revalidate it against the current target catalog. +Examples that omit an explicit contribution target rely on legacy singleton +resolution. Before adapting one for submission, add the appropriate +`[competition]` table and revalidate it against the registered target catalog. ## Positive learning examples @@ -37,7 +37,7 @@ These bundles are meant to fail or expose a gate: | [`miner_rmsnorm_broken`](https://github.com/latent-to/optima/tree/main/examples/miner_rmsnorm_broken) | an incorrect normalization is rejected despite plausible output shape | | [`miner_setup_demo`](https://github.com/latent-to/optima/tree/main/examples/miner_setup_demo) | legacy engine-wide `setup` surface for isolation tests | -`miner_setup_demo` is not a registered component template. No current registered +`miner_setup_demo` is not a registered component template. No registered target permits `setup`; a cross-cutting engine change belongs in [Discovery](discovery-lane.md). @@ -47,11 +47,11 @@ target permits `setup`; a cross-cutting engine change belongs in demonstrates a Torch reference paired with a CuTe-DSL epilogue declaration for `moe.fused_experts/gemm1_epilogue`. -The composition and CPU/dense path work. The validator-owned GPU base kernel is -currently a stub, so this example cannot reproduce a GPU win through the -current override path. Read [Override points](override-points.md) before using -it, and do not treat historical provenance fields in its metadata as a result -of this bundle. +This example exercises composition and the CPU/dense reference path. The +validator-owned GPU base kernel raises `NotImplementedError`, so the example is +not eligible for GPU qualification through that override point. Read +[Override points](override-points.md) before using it. Provenance fields in its +metadata are descriptive inputs, not evidence produced by this bundle. ## Identity fixtures are not kernels diff --git a/docs/miner-guide/finding-a-win.md b/docs/miner-guide/finding-a-win.md index 4a2789b..934faf1 100644 --- a/docs/miner-guide/finding-a-win.md +++ b/docs/miner-guide/finding-a-win.md @@ -94,11 +94,10 @@ For attention work: - an optimization to one should not be measured on a workload dominated by the other. -The developer `evaluate --input-len` option can generate longer inputs, but -repeated timed prompts with radix caching enabled may turn later passes into -cache hits. For a prefill diagnostic, use the arena's cache policy or explicitly -disable the radix cache in both arms. Treat this as local debugging, not -authority. +A developer diagnostic can generate longer inputs, but repeated timed prompts +with radix caching enabled may turn later passes into cache hits. For a prefill +diagnostic, reproduce the arena's cache policy or explicitly disable the radix +cache in both arms. Treat this as local debugging, not authority. For collective work, reproduce the actual world/TP size and interconnect. A single-GPU benchmark says nothing about distributed ordering, graph replay, or @@ -120,9 +119,9 @@ T = separately defined pristine quality reference ``` Changing a backend, graph flag, prompt mix, topology, or engine option only in C -does not measure a component delta. The local `evaluate` CLI exposes such flags -for diagnosis, but authoritative qualification binds all four arms from the -validator's stack and reference manifests. +does not measure a component delta. Ad hoc developer launches may expose such +controls for diagnosis, but authoritative qualification binds all four arms +from the validator's stack and reference manifests. The repository implementations are [`stack_plan.py`](https://github.com/latent-to/optima/blob/main/optima/stack_plan.py) @@ -152,9 +151,9 @@ Write a small target worksheet before implementation: | local correctness proof needed | all applicable slot shapes, mutation/output checks, CUDA replays | | end-to-end falsifier | no throughput change, B/B′ drift, fallback on material calls, or quality regression | -This forces the optimization story to connect all the way from a measured bottleneck to -a selectable variant. “My kernel is 30% faster” is incomplete until you can state what -fraction of which arena path it replaces and how often the domain routes. +This worksheet requires a causal chain from the measured bottleneck to a selectable +variant. “My kernel is 30% faster” is incomplete until the replaced fraction of the arena +path and the domain's routing frequency are specified. ## Use a disciplined development ladder @@ -164,8 +163,10 @@ fraction of which arena path it replaces and how often the domain routes. 3. Microbenchmark the exact callable to understand the mechanism, not to claim end-to-end speedup. 4. Trace the complete incumbent and candidate developer launches. -5. Run graph-on `evaluate` with matched controls and inspect B/C/B′ drift. -6. Run the relevant `bench`/quality diagnostics. +5. Run a matched graph-on whole-engine diagnostic using the + [canonical local profiling record](../validator-guide/running-evals.md#performance-development) + and inspect B/C/B′ drift. +6. Run the relevant non-authoritative quality diagnostics. 7. Repeat across fresh processes, seeds, and conditioning until the effect is larger than normal variance. diff --git a/docs/miner-guide/graph-safety.md b/docs/miner-guide/graph-safety.md index dd343ad..3908ab4 100644 --- a/docs/miner-guide/graph-safety.md +++ b/docs/miner-guide/graph-safety.md @@ -155,10 +155,10 @@ arena requirement before investing in an eager-only specialization. ## Do not benchmark a different regime -`evaluate --disable-cuda-graph` exists for debugging startup or math. It changes -the incumbent execution regime and therefore cannot establish a production -speedup. A graph-on candidate must be compared with the graph-on incumbent under -the same evaluation stack. +Disabling CUDA graphs in a contributor-controlled launch can help debug startup or math, +but it changes the incumbent execution regime and therefore cannot establish a production +speedup. A graph-on candidate must be compared with the graph-on incumbent under the same +evaluation stack. When graph verification fails, use the stage-specific guidance in [Diagnostics](diagnostics.md). Do not work around it by narrowing metadata until diff --git a/docs/miner-guide/kernel-abi.md b/docs/miner-guide/kernel-abi.md index c676049..2cfcd90 100644 --- a/docs/miner-guide/kernel-abi.md +++ b/docs/miner-guide/kernel-abi.md @@ -57,6 +57,71 @@ The authoritative ABI objects are in output shape/stride checks in [tensor_spec.py](https://github.com/latent-to/optima/blob/main/optima/tensor_spec.py). +## Direct-artifact form of the ABI + +A sealed direct artifact implements the same slot semantics without a runtime +Python callable. Its `bindings` list projects the immutable slot call ABI into a +closed native signature. For example, a simple tensor kernel can bind input and +output device pointers plus the validator's current stream: + +```toml +bindings = [ + { source = "input.x", kind = "pointer", projection = "device_ptr" }, + { source = "output.out", kind = "pointer", projection = "device_ptr" }, + { source = "stream.current", kind = "stream" }, +] +``` + +The binding order is the index space used by the device plan. It is not +necessarily the CUDA parameter order: one semantic tensor binding can feed a +pointer, dimension expression, stride expression, packed field, and TMA +descriptor. The validator joins every expression and parameter reference back to +the typed binding before prebuild and again when reopening sealed state. + +The complete direct ABI has three parts: + +| Layer | Declared by the bundle | Owned at runtime by the validator | +|---|---|---| +| Semantic call | Ordered projections of registered slot resources | Live tensors, scalars, stream, group, and output ownership | +| Artifact storage | Bounded `workspace.*`, `prepared.*`, and `state.*` rows | Allocation, address, scope, budget, and lifecycle | +| Device launch | Complete logical kernels, parameter widths, and ordered launch plans | Physical CUBIN admission, parameter packing, TMA/FastDivmod construction, launch, and cleanup | + +Device parameters are a closed set: exact scalars, admitted pointers, packed +structs with checked non-overlapping fields, 128-byte TMA descriptors, +CuTe/CUTLASS FastDivmod values, and provider-authorized group handles. Checked +expressions can read live tensor shape, stride, element size, storage offset, +element count, and admitted scalars. They cannot execute candidate callbacks, +construct arbitrary pointer arithmetic, or discover an ambient stream or group. + +The manifest declares logical kernel names because CuTe-generated physical names +depend on the materialized module. After rank CUDA setup, the validator observes +the exact sealed CUBIN through the Driver API and binds the canonical logical +inventory to the physical inventory by ordinal. Kernel count and every formal +parameter width must match. The admitted library handle is retained for launch, +so inspection and execution refer to the same loaded object. + +Lifecycle roles are ordered `init`, `prepare`, `reset`, `run`, `destroy`. +`workspace.*` is call-local; `prepared.*` must cross prepare to run; `state.*` +persists with the engine artifact entry. A `prelaunch` fill can initialize an +authorized output or artifact resource, but there is no general host prelaunch +callback. + +Direct execution still obeys capability routing, output poisoning, graph replay, +reference comparison, full-engine quality, and reproduction. Qualification also +requires `aot_loaded`, `aot_invoked`, and normal `completed` receipts from every +active scheduler member, with no fallback receipt. Loading a CUBIN is not proof +that it provided the measured result. + +Group-aware projections are declaratively recognized only through the supplied +slot group and exact persistent `group_ipc` resources. This schema support does +not make the standard CuTe load path executable: it supplies no group +capability/handle resolvers, so these projections fail closed. A reviewed resolver +integration would still need distributed evidence for each concrete topology and +plan. + +See [Sealed direct artifacts](../architecture/direct-artifacts.md) and the +[manifest field reference](../reference/manifest-schema.md). + ## Op slots ### `activation.silu_and_mul` diff --git a/docs/miner-guide/override-points.md b/docs/miner-guide/override-points.md index ca8ddd3..3e54008 100644 --- a/docs/miner-guide/override-points.md +++ b/docs/miner-guide/override-points.md @@ -8,9 +8,9 @@ serving integration remain validator-owned. This is an advanced composition mechanism, not permission to patch arbitrary engine code. -## Current scope and status +## Registered capability -There is currently one implemented override-point identity: +The override registry defines one override-point identity: ```text moe.fused_experts/gemm1_epilogue @@ -20,19 +20,19 @@ It selects the `gemm1_epilogue` hole in the validator-owned `nvfp4_moe_megakernel` base. The registry and composition code are in [override.py](https://github.com/latent-to/optima/blob/main/optima_kernels/override.py). -The status matters: +Its support boundary is explicit: - manifest parsing, point lookup, composition, and the dense/CPU reference path are implemented and tested; -- the GPU `nvfp4_moe_megakernel.run` path is currently a stub that raises - `NotImplementedError`; -- therefore the committed example is useful for ABI development but cannot - establish a GPU crown through this base today. +- the validator-owned GPU `nvfp4_moe_megakernel.run` path raises + `NotImplementedError`; and +- therefore the committed example supports ABI development but is not eligible + for GPU qualification through this base. See the explicit status in [nvfp4_megakernel.py](https://github.com/latent-to/optima/blob/main/optima_kernels/moe/nvfp4_megakernel.py). -Do not quote provenance measurements embedded in example metadata as results of -the current override bundle. +Provenance measurements embedded in example metadata are not evidence produced by +the override bundle. ## Bundle declaration @@ -92,16 +92,16 @@ cute` or the equivalent absolute `from cutlass import cute`. Rebinding or shadowing the alias, importing it twice, using another module's `.compile`, vendoring a local `cutlass` package, or passing a string/bytes literal as the first positional argument fails closed. The validator-owned -`DSL_JIT_ENTRYPOINTS` table contains only CuTe today. The exception exists only +`DSL_JIT_ENTRYPOINTS` table admits only CuTe. The exception exists only for CuTe's function-object tracing API; it does not permit string-to-code compilation or `builtins.compile`. Triton's lazy `@triton.jit` launch path does not need an explicit `.compile` admission. -The current admission logic is pinned in the post-main working-tree paths -`optima/dsl_jit_policy.py` and `optima/engine_tree.py` at the commit named in -[State of record](../reference/state-of-record.md). +The authoritative admission logic is +[`dsl_jit_policy.py`](https://github.com/latent-to/optima/blob/main/optima/dsl_jit_policy.py) +and [`engine_tree.py`](https://github.com/latent-to/optima/blob/main/optima/engine_tree.py). -The exact device ABI belongs to the registered point. For the current point, +The exact device ABI belongs to the registered point. For this point, `alpha_val` is the per-expert quantization/dequantization value; activation parameters such as a sigmoid gain are distinct values. Mixing those two meanings is a correctness bug. diff --git a/docs/miner-guide/overview.md b/docs/miner-guide/overview.md index 8fdd6d7..7cba4bd 100644 --- a/docs/miner-guide/overview.md +++ b/docs/miner-guide/overview.md @@ -130,20 +130,19 @@ and [Discovery lane](discovery-lane.md). ## Your development loop -The local CLI gives you useful diagnostics: +The local CLI provides static and component diagnostics: ```bash python -m optima.cli scan my_bundle python -m optima.cli verify my_bundle --device cuda --dtype bfloat16 -python -m optima.cli evaluate my_bundle --model ... -python -m optima.cli bench my_bundle --model ... ``` -These commands help find manifest, ABI, correctness, graph, fidelity, and -performance problems. They do **not** reproduce crown authority: a local run -does not possess the finalized intake record, validator stack manifest, hidden -inputs, calibrated policies, immutable publications, isolated service, or -second independent qualification attempt. +These commands find manifest, static-policy, ABI, correctness, routing, and graph +problems. Profile and bracket serving performance in an environment matching the +published arena contract. Neither the CLI checks nor a contributor-controlled A/B run +reproduces crown authority: local work does not possess the finalized intake record, +validator stack manifest, hidden inputs, calibrated policies, immutable publications, +isolated service, or second independent qualification attempt. Use this guide in order: diff --git a/docs/miner-guide/slots.md b/docs/miner-guide/slots.md index 84bd483..5791ca4 100644 --- a/docs/miner-guide/slots.md +++ b/docs/miner-guide/slots.md @@ -38,9 +38,15 @@ manifest; it does not require the Python function itself to be named `entry`. | `collective.moe_finalize_ar_rmsnorm` | collective | `entry(gemm_out, row_map, scales, residual, weight, eps, out_norm, out_residual, group)` | finalized/reduced residual and normalized output | The two MSA slots output **scores**, not selected indices or attended values. -Their target contracts grade the validator-owned top-k result. The prefill slot -is a distinct long-prefill boundary and its canonical live descriptor is eager; -do not assume a decode optimization exercises it. +The production engine's runtime top-k controls the validator-owned selection tail. +`SlotSpec.correctness.top_k` is a separate verification parameter: it selects the +width of the score-sheet overlap comparison (the registered width is eight blocks), but does not +configure production selection or gate whether the score kernel routes. A runtime +call may therefore use a different top-k while the candidate still fills the complete +score sheet and the stock tail applies the engine's value. + +The prefill slot is a distinct long-prefill boundary and its canonical live +descriptor is eager; do not assume a decode optimization exercises it. Registration and live installation are also different facts. The `attention.msa_block_score` decode contract is registered and CPU-verifiable, but its diff --git a/docs/miner-guide/submitting.md b/docs/miner-guide/submitting.md index fcdc139..96a9ece 100644 --- a/docs/miner-guide/submitting.md +++ b/docs/miner-guide/submitting.md @@ -62,7 +62,7 @@ development checks appropriate to the target: ```bash python -m optima.cli scan my_bundle -python -m optima.cli verify my_bundle --device cuda --dtype bfloat16 ... +python -m optima.cli verify my_bundle --device cuda --dtype bfloat16 ``` Inspect the tree for credentials, caches, generated binaries, model data, @@ -85,9 +85,9 @@ if it changes while the hosted archive does not, the validator will reject the fetch as a content mismatch. For extra confidence, extract the hosted object into a clean temporary location and run -`chain-package` or the bundle hash on that extracted root. The wrapper directory and gzip -encoding are not the identity; the sorted relative paths and file bytes are. Never -“refresh” a stable URL with revised content after committing the old hash. +`chain-package` against that root, then compare its printed content hash. The wrapper +directory and gzip encoding are not the identity; the sorted relative paths and file bytes +are. Never “refresh” a stable URL with revised content after committing the old hash. ## 3. Host over stable public HTTPS @@ -232,12 +232,12 @@ Use the operator's designated status surface rather than assuming absence from A crown remains separate from source integration and an Optima Engine release. Submitting does not cause the validator to publish miner code as a release. -## Do not use the legacy local ledger as production +## Production authority boundary -The top-level `commit`, `reveal`, `ledger`, and `legacy-settle` commands operate -the development JSON simulation. They are useful for tests and demonstrations, -but they do not create finalized chain intake, production qualification, -transactional settlement, or weights. +The supported submission path is `chain-package` followed by `chain-submit`. Finalized +chain intake, SQLite qualification state, transactional settlement, and journaled weight +publication are separate validator authorities. No local ledger or contributor-side +score can create those records. If the proposal stalls or fails, map its reported lifecycle state through [Diagnostics](diagnostics.md). diff --git a/docs/miner-guide/your-first-kernel.md b/docs/miner-guide/your-first-kernel.md index d49551b..cfb5951 100644 --- a/docs/miner-guide/your-first-kernel.md +++ b/docs/miner-guide/your-first-kernel.md @@ -146,27 +146,25 @@ python -m optima.cli verify my_collective \ --device cuda --dtype bfloat16 --world-size 4 --tp-size 4 ``` -Then use the engine-facing developer diagnostics while iterating: - -```bash -python -m optima.cli evaluate my_silu \ - --model --tp-size ... - -python -m optima.cli bench my_silu \ - --model --tp-size \ - --benchmarks ... +Then follow the canonical +[performance-development procedure](../validator-guide/running-evals.md#performance-development) +in an environment matching the published arena contract. No repository command +materializes the incumbent/candidate engines for this local experiment. Bracket the +candidate with identical incumbent runs: + +```text +B -> C -> B′ +speedup = candidate_rate / mean(baseline_before_rate, baseline_after_rate) ``` -`evaluate` and `bench` are **developer diagnostics, not crown authority**. Their -flags are intentionally useful for debugging, including candidate-only launch -changes. An authoritative candidate cannot choose a different evaluation stack; -the validator binds B, C, B′, T, resources, graph evidence, hidden inputs, and -calibrated policies itself. +Keep the arena's CUDA-graph state, topology, model, dtype, workload, and charged-work +definition fixed. Reject a result when B/B′ drift is comparable to the claimed gain. +For a long-prefill target, make the workload genuinely prefill-heavy; repeated prompts +with a live radix cache can silently turn later iterations into decode/cache-hit work. -Never call a graphs-off result a win. `--disable-cuda-graph` is a debugging -mode that changes the incumbent regime. For a long-prefill target, also make the -workload genuinely prefill-heavy; repeated prompts with a live radix cache can -silently turn later iterations into decode/cache-hit measurements. +This local bracket is a performance hypothesis, not crown authority. The validator binds +B, C, B′, pristine T, resources, graph evidence, hidden inputs, and calibrated policies, +then requires a separately bound reproduction. ## 7. Decide whether the target is worth pursuing diff --git a/docs/reference/arenas.md b/docs/reference/arenas.md index 796bd03..a4e5c19 100644 --- a/docs/reference/arenas.md +++ b/docs/reference/arenas.md @@ -74,7 +74,7 @@ The provider protocol has only two authority-bearing operations: ```python def run_screen(manifest, stage, candidate) -> ScreenStageResult: ... -def build_qualification(request) -> ArenaQualificationWork: ... +def build_qualification(request, state=None) -> ArenaQualificationWork: ... ``` `ArenaService` checks the provider digest on construction, requires the exact @@ -112,13 +112,18 @@ Admission has three typed outcomes before evidence exists: |---|---|---| | `ADMIT` | Screen/qualification capacity is available | Reserve work under the bound service | | `QUEUE` | The relevant active-worker capacity is temporarily full | Preserve finalized priority and retry later | -| `HOLD` | Queue depth/age, cohort size, or retry budget crosses policy | Stop automatic progress pending operator/policy handling | +| `HOLD` | Queue depth/age or cohort size crosses admission policy | Stop automatic progress pending operator/policy handling | Screen and qualification capacities are distinct. A cohort larger than `max_cohort_size` is held; a cohort that fits policy but would exceed current active qualification capacity is queued. This avoids converting overload into candidate failure. +Retry exhaustion is post-evidence policy, not an admission input. After a screen or +qualification produces `NO_DECISION`, `retry_disposition()` returns the separate +`PromotionDecision.RETRY` while the lane-specific attempt is below budget, otherwise +`PromotionDecision.HOLD`. + ## Screen receipt derivation The screen receipt must be a canonical prefix of the fixed stage order. Every @@ -156,7 +161,9 @@ evaluator's summary text. ## Deployment review checklist - Construct the registry from a sorted, unique, reviewed arena list. -- Pin provider code by the digest embedded in each service manifest. +- Independently attest the reviewed installed provider bytes/configuration to the + `provider_digest` embedded in each service manifest. `ArenaService` compares that digest + with the provider object's self-declared value; the type boundary is not byte attestation. - Keep model paths, executor handles, entropy sources, and hidden judges inside deployment code; expose only their typed identities at the boundary. - Ensure queue snapshots and leases come from the durable single-writer diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 79eb382..3d6897e 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,48 +1,41 @@ # CLI reference -The CLI exposes three different surfaces: production control-plane operations, -local contribution tools, and development-only simulations. They share an -entrypoint but do **not** share authority. +The CLI is a thin operator and contributor surface over Optima's typed APIs. A +command's output has only the authority listed here; human-readable output is not a +qualification, settlement, or release receipt. ```bash python -m optima.cli [options] python -m optima.cli --help ``` -Using the module form is recommended on GPU hosts because SGLang starts child -processes. The installed `optima` console script targets the same parser. +Use module invocation on GPU hosts because SGLang starts child processes. The +installed `optima` console script resolves to the same parser. ## Command inventory | Command | Audience | Authority | Purpose | |---|---|---|---| -| `slots` | all | read-only | Print the live slot ABI | -| `compat` | operator | diagnostic | Check the installed SGLang seam against the pin | -| `chain-compat` | operator | diagnostic | Check the installed Bittensor SDK surface without network access | -| `scan` | contributor | local gate | Parse a bundle and apply static policy | -| `verify` | contributor | local gate | Check the declared slot contract against a trusted reference | -| `hash` | contributor | identity | Print the deterministic bundle content hash | -| `evaluate` | developer | **development diagnostic** | Run the older direct throughput/fidelity harness | -| `bench` | developer | **development diagnostic** | Run direct task/throughput diagnostics | -| `chain-package` | contributor | submission | Build the deterministic archive to host | -| `chain-submit` | contributor | submission | Commit a bundle hash and HTTPS fetch location | -| `chain-status` | all | read-only | Inspect current subnet and reveal state | -| `chain-register` | operator | chain mutation | Register a hotkey and run its preflight | -| `chain-validate` | validator | production | Consume finalized reveals; optionally run an injected arena | -| `set-weights` | validator | production | Reconcile the journaled global reward projection | -| `model-provision` | release operator | production | Seal model bytes into a content-addressed external receipt | -| `release-verify` | release consumer | production | Verify and reopen a signed Engine release | -| `release-context` | release consumer | production | Produce a deterministic OCI build context from a verified release | -| `commit`, `reveal`, `ledger` | developer | local simulation | Exercise the legacy local JSON commit/reveal model | -| `legacy-settle` | developer | **development-only** | Run the old settlement simulation; requires `--development-only` | - -!!! warning "Know which path you are running" - `evaluate`, `bench`, and `legacy-settle` are useful diagnostics. They are - not the production qualification or settlement authority and cannot mint a - current crown. The hardened path is a registered arena injected into - `chain-validate`. - -## Local contribution loop +| `slots` | all | read-only | Print the registered slot ABI | +| `compat` | operator | diagnostic | Check the installed SGLang seam surface against the pin | +| `chain-compat` | operator | diagnostic | Check the installed Bittensor SDK surface without chain access | +| `scan` | contributor | local gate | Parse a bundle and apply recursive static policy | +| `verify` | contributor | local gate | Check declared slot behavior against validator-owned references | +| `chain-package` | contributor | packaging | Build a canonical archive and print its content hash | +| `chain-submit` | contributor | chain mutation | Commit a bundle hash and HTTPS fetch location through timelock reveal | +| `chain-status` | all | read-only | Inspect public subnet, registration, and reveal state | +| `chain-register` | operator | chain mutation | Burn-register a hotkey and run the SDK preflight | +| `chain-validate` | validator | production intake | Consume finalized reveals; a deployment may inject qualification services | +| `set-weights` | validator | production control plane | Reconcile the journaled global reward projection | +| `model-provision` | release operator | production artifact | Seal model bytes into a content-addressed publication and receipt | +| `release-verify` | release consumer | production verification | Reopen a signed Engine release under an externally trusted key | +| `release-context` | release consumer | production build input | Materialize a deterministic OCI context from a verified release | + +There is no local command that grants qualification or settlement authority. Complete +engine qualification begins at the deployment-injected arena boundary; a local evaluator +or JSON ledger is not an alternate production interface. + +## Contribution commands ### `slots` @@ -50,8 +43,8 @@ processes. The installed `optima` console script targets the same parser. python -m optima.cli slots ``` -Reads the live `SLOTS` registry. It does not import contribution code or require -a GPU. See the [slot catalog](slots-table.md). +Reads the registered `SLOTS` table without importing contribution code or requiring a +GPU. See the [slot catalog](slots-table.md). ### `scan` @@ -59,54 +52,49 @@ a GPU. See the [slot catalog](slots-table.md). python -m optima.cli scan path/to/bundle ``` -Loads `manifest.toml` as data and checks the complete declared tree. A clean AST -scan is defense in depth; it does not make contribution code trusted. +Loads `manifest.toml` as data and recursively applies the Python policy to declared and +vendored `.py` files. Manifest-declared CUDA sources and dependency patches are admitted +as separate reviewed-build tiers; undeclared executable files, binaries, and symlinks are +rejected. Static scanning is defense in depth; a clean result does not make contribution +code trusted. ### `verify` ```bash -# CPU sanity check +# CPU contract smoke python -m optima.cli verify examples/miner_silu_torch \ --device cpu --dtype float32 -# CUDA slot verification +# CUDA contract and graph verification python -m optima.cli verify path/to/bundle \ --device cuda --dtype bfloat16 --model MiniMax-M3 -# Collective verification at the arena topology +# Distributed verification at the arena topology python -m optima.cli verify path/to/bundle \ --device cuda --world-size 4 ``` -Useful options are `--dtype`, `--device`, `--seed`, `--world-size`, `--tp-size`, -and `--model`. Verification proves only the registered component contract. It -does not prove a serving speedup or grant settlement authority. +Options are `--dtype`, `--device`, `--seed`, `--world-size`, `--tp-size`, and +`--model`. The verifier rejects ambiguous variant domains, applies the model-specific +slot profile, and spawns workers for candidate execution. Collective slots use the +requested rank count; a host without enough CUDA devices falls back to CPU/Gloo unless +`--device cuda` makes the requirement explicit. -### `evaluate` and `bench` - -Both commands launch direct developer diagnostics and expose many experimental -knobs. Always inspect their live help: - -```bash -python -m optima.cli evaluate --help -python -m optima.cli bench --help -``` - -Do not use graphs-off measurements as performance evidence. Do not translate a -direct diagnostic result into a production PASS, reproduction, or crown. +Verification proves only the exercised component contract. It does not establish model +integration, serving throughput, pristine quality, isolation, independent reproduction, +or settlement. ## Submission commands -### Package and hash +### Package ```bash python -m optima.cli chain-package path/to/bundle --out bundle.tar.gz -python -m optima.cli hash path/to/bundle ``` -Host the archive at a stable **HTTPS** URL. The commitment binds the canonical -bundle content hash; validators fetch and independently rehash the extracted -tree. +The command writes one canonical wrapper archive and prints the deterministic content +hash of the extracted bundle tree. Host the exact archive at a stable HTTPS URL. The URL +is transport; the hash is proposal identity. ### Submit @@ -118,54 +106,54 @@ python -m optima.cli chain-submit path/to/bundle \ --blocks-until-reveal 10 ``` -Use `--dry-run` to construct and print the payload without signing or sending -it. A submitted commitment is unreadable until its reveal delay elapses; the -validator acts only on finalized, valid reveals. +`chain-submit` re-hashes the local bundle before constructing the payload. Use +`--dry-run` to print the payload without signing or submitting it. Validators act only +on finalized, valid reveals and independently fetch, extract, and re-hash the hosted +archive. -### Inspect chain state +### Inspect public chain state ```bash python -m optima.cli chain-status \ - --network --netuid + --network --netuid \ + --wallet --hotkey ``` -Adding `--wallet` and `--hotkey` reports registration information for that -identity. +The wallet arguments add registration information. This command does not expose the +validator's private intake, screening, qualification, or settlement database. -## Validator commands - -### `chain-validate` - -Intake-only mode is complete and intentionally available without a GPU arena: +### Register a hotkey ```bash -python -m optima.cli chain-validate \ +python -m optima.cli chain-register \ --network --netuid \ - --intake-only --once + --wallet --hotkey ``` -It performs finalized ordering, hardened fetch, rehashing, private retention, -and immutable worker publication. Full qualification additionally requires -deployment code to inject an `ArenaServiceRegistry` and select its closed -service with `--arena-id`. A reviewed deployment wrapper calls -`cmd_chain_validate(args, arena_registry=registry)` (or `run_validator(...)`) -after constructing that registry. Its operator-facing invocation may retain -the same arguments: +This is a chain mutation and may burn registration cost. It checks for existing +registration first, then runs the SDK preflight. + +## Validator commands + +### `chain-validate` + +The stock entrypoint supports complete finalized intake without a GPU service: ```bash python -m optima.cli chain-validate \ --network --netuid \ - --arena-id + --intake-only --once ``` -The shell command above is **not sufficient through the stock entrypoint**: -`python -m optima.cli` has no way to inject the registry and will refuse it. -The repository does not ship a production `ArenaServiceProvider`, and the -documentation does not provide a shell callback escape hatch. +Intake mode persists finalized order, hardened fetch and re-hash results, private +retention, immutable worker publication, and copy disposition. Storage and loop controls +are `--intake-db`, `--private-root`, `--publication-root`, `--interval`, and `--once`. -Storage and loop controls are `--intake-db`, `--private-root`, -`--publication-root`, `--interval`, and `--once`. These paths contain durable -authority; back them up and apply the documented permissions. +Qualification requires a deployment-owned `ArenaServiceRegistry` plus a registered +`--arena-id`. The repository does not construct a production provider from shell text. +A reviewed deployment wrapper calls `cmd_chain_validate(args, +arena_registry=registry)` or `run_validator(...)` after creating the registry. Invoking +the stock module without `--intake-only` refuses to run because no registry was injected. ### `set-weights` @@ -179,12 +167,25 @@ python -m optima.cli set-weights \ --dry-run ``` -The command recomputes a pure global projection from settled state. With -`--dry-run`, it returns the proposed payload without creating a journal intent -or submitting. Without `--dry-run`, reconciliation records the intent in the -SQLite journal and may sign/submit it. An SDK success return leaves the intent -pending; later chain observation is what confirms it. `--release-hold REASON` -appends an audited release of a held publication and does not submit it. +The command reopens settled state and computes a pure global projection. `--dry-run` +creates no publication intent and submits nothing; stable stdout reports the projection +digest, `status=dry_run`, `chain_matches`, and `submitted=False`. It does not print the +projected UID/weight vector or the complete projection inputs. A live reconciliation +journals intent before submission and confirms it only through later chain observation. +`--release-hold REASON` appends an audited release of the held publication and does not +submit. + +## Environment checks + +```bash +python -m optima.cli compat +python -m optima.cli chain-compat +``` + +`compat` checks imports and signatures for the pinned SGLang seam. A version mismatch is +reported but is not, by itself, a failing exit status; automation must compare the +reported version to `PINNED_SGLANG`. `chain-compat` checks the Bittensor SDK API used by +Optima without connecting to a network. ## Release commands @@ -193,11 +194,10 @@ appends an audited release of a held publication and does not submit it. ```bash python -m optima.cli model-provision \ /srv/models/model /srv/optima/model-publication \ - --expected-content-digest + --expected-content-digest --workers ``` -The result is an immutable content-addressed tree plus a receipt. `--workers` -controls hashing parallelism. +The result is an immutable content-addressed model tree and receipt. ### Verify a release @@ -207,28 +207,29 @@ python -m optima.cli release-verify /srv/optima/releases/ \ --descriptor-digest ``` -Release verification requires an Optima installation with the `release` extra. - -The expected public key is an external trust input; accepting a key discovered -inside the release would not authenticate its signer. +The expected public key is an external trust input. A key discovered only inside the +release cannot authenticate its signer. ### Materialize a container context ```bash -python -m optima.cli release-context /srv/optima/releases/ ./context \ +python -m optima.cli release-context \ + /srv/optima/releases/ ./context \ --expected-public-key \ --descriptor-digest ``` -This first reopens the complete signed release and then emits a deterministic -OCI context. Release construction and signing are programmatic APIs; there is -no public `release-create` command. +The command reopens the complete signed publication before writing a deterministic OCI +context. Release construction and signing remain programmatic APIs; there is no public +`release-create` command. ## Exit behavior -Most commands return `0` on success and a non-zero code on refusal or failed -checks. Scripts should rely on the process status and retain structured/durable -receipts where a command emits them. Human-readable text is not settlement -evidence. +Commands use a non-zero status for parser errors, explicit refusals, and failed local +checks, but exit `0` is only command completion. It is not always business-state success: +for example, `set-weights` can complete with a durable `pending` publication that still +requires later chain observation. Automation must inspect the typed status and retain the +receipts or durable records emitted by the authoritative subsystem. Console prose and +process status alone are never settlement or publication evidence. Source: [`optima/cli.py`](https://github.com/latent-to/optima/blob/main/optima/cli.py). diff --git a/docs/reference/codebase-map.md b/docs/reference/codebase-map.md index c31b732..0f9417f 100644 --- a/docs/reference/codebase-map.md +++ b/docs/reference/codebase-map.md @@ -13,6 +13,7 @@ are trying to understand: ```mermaid flowchart LR M["manifest + slots + target catalog"] --> D["local dispatch and verification"] + M --> N["sealed artifact prebuild + device runtime"] M --> I["finalized intake + immutable publication"] I --> A["injected arena service"] A --> Q["isolated qualification evidence"] @@ -20,11 +21,16 @@ flowchart LR S --> W["emissions + weight journal"] Q --> R["integration review"] R --> E["Engine tree + signed release"] + N --> Q + N --> E ``` -The left branch is useful to contributors but cannot crown anything. The upper -production branch owns hostile evaluation and economic state. The release -branch reopens evidence but accepts only reviewed integrated source. +The local branch is useful to contributors but cannot crown anything. The +intake, arena, qualification, settlement, and weight branch owns hostile +evaluation and economic state. A sealed direct artifact enters qualification +through the registered prebuild/runtime boundary; after integration review, its +sealed native publication may also be bound into a release. The release branch +reopens evidence but accepts only reviewed integrated source. ## Contribution contract @@ -35,10 +41,25 @@ branch reopens evidence but accepts only reviewed integrated source. | Target identity and composition | [`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py) | | Typed tensor/output boundary | [`tensor_spec.py`](https://github.com/latent-to/optima/blob/main/optima/tensor_spec.py) | | Static policy | [`sandbox.py`](https://github.com/latent-to/optima/blob/main/optima/sandbox.py) | -| Tracing-JIT admission (post-main snapshot) | `optima/dsl_jit_policy.py` | +| Tracing-JIT admission | [`dsl_jit_policy.py`](https://github.com/latent-to/optima/blob/main/optima/dsl_jit_policy.py) | | Local and distributed verification | [`verify.py`](https://github.com/latent-to/optima/blob/main/optima/verify.py), [`verify_collective.py`](https://github.com/latent-to/optima/blob/main/optima/verify_collective.py) | | SGLang dispatch | [`dispatch.py`](https://github.com/latent-to/optima/blob/main/optima/dispatch.py), [`seams.py`](https://github.com/latent-to/optima/blob/main/optima/seams.py) | -| Scheduler-role candidate load (post-main snapshot) | `optima/seam.py`, `optima/integrations/sglang_scheduler_gate.py` | +| Scheduler-role candidate load | [`seam.py`](https://github.com/latent-to/optima/blob/main/optima/seam.py), [`sglang_scheduler_gate.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_scheduler_gate.py) | + +## Sealed direct artifacts + +| Area | Primary source | +|---|---| +| Closed provider policy | [`artifact_provider.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_provider.py) | +| Slot call ABI, resources, and lifecycle | [`artifact_abi.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_abi.py), [`artifact_runtime.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_runtime.py) | +| Canonical direct-execution identity | [`artifact_identity.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_identity.py), [`artifact_resource_identity.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_resource_identity.py) | +| Declarative device launch | [`artifact_device_launch.py`](https://github.com/latent-to/optima/blob/main/optima/artifact_device_launch.py) | +| CUBIN ABI and Driver admission | [`cuda_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/cuda_cubin.py), [`cuda_launch.py`](https://github.com/latent-to/optima/blob/main/optima/cuda_launch.py) | +| Parameter, TMA, and FastDivmod materialization | [`cuda_materialize.py`](https://github.com/latent-to/optima/blob/main/optima/cuda_materialize.py) | +| CuTe compiler boundary and sealed index | [`cute_aot.py`](https://github.com/latent-to/optima/blob/main/optima/cute_aot.py), [`cute_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/cute_cubin.py) | +| Measured compile profile | [`eval/native_compile_profile.py`](https://github.com/latent-to/optima/blob/main/optima/eval/native_compile_profile.py) | +| Registered build patcher | [`patchers/build_cute_cubin.py`](https://github.com/latent-to/optima/blob/main/optima/patchers/build_cute_cubin.py) | +| Rank-local post-CUDA binding | [`integrations/sglang_artifact_context.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_artifact_context.py) | ## Intake and referee @@ -54,6 +75,7 @@ branch reopens evidence but accepts only reviewed integrated source. | Host-owned B/C/B′ session | [`eval/oci_outer_session.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_outer_session.py) | | Immutable native prebuild | [`eval/oci_prebuild.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_prebuild.py) | | Device conditioning/cleanup | [`eval/device_state.py`](https://github.com/latent-to/optima/blob/main/optima/eval/device_state.py) | +| Compile-profile and multi-architecture prebuild | [`eval/native_compile_profile.py`](https://github.com/latent-to/optima/blob/main/optima/eval/native_compile_profile.py), [`eval/oci_prebuild.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_prebuild.py) | ## State, economics, and weights @@ -93,8 +115,9 @@ Read in this order: 1. `manifest.py` for exact TOML shape and contained-path rules; 2. `sandbox.py` and `dep_policy.py` for observed source/build features; 3. `target_catalog.py` for target resolution and admitted features; -4. `slots.py` and `tensor_spec.py` for callable/output semantics; and -5. `verify.py` or `verify_collective.py` for executable correctness. +4. `artifact_provider.py` and `artifact_abi.py` when the row declares direct exports; +5. `slots.py` and `tensor_spec.py` for callable/output semantics; and +6. `verify.py` or `verify_collective.py` for executable correctness. This ordering separates syntax, capability admission, ABI, and numerical failure. They are different diagnoses even when the CLI reports them in one @@ -118,7 +141,8 @@ Console output and evaluator summary text are never the settlement input. Read `economics.py` first: it computes the pure global projection from reopened active contribution state. Then read `chain/weights.py`: it refreshes the live metagraph, journals intent before submission, records SDK results without -treating them as confirmation, and reconciles later chain observation. +treating them as confirmation, and reconciles later chain observation. The journal stores +status/chronology metadata and a projection digest, not raw pre/post readback vectors. `--dry-run` creates no journal intent, while reconciliation and submission have different durable states. @@ -136,9 +160,9 @@ materialization owns deterministic module namespaces and packaging. Finally, |---|---|---| | Parsed bundle | Contributor/diagnostic process | Content hash over the admitted bundle tree | | Finalized publication | Intake controller | Validator-owned immutable worker publication | -| Qualification evidence | External evidence root | Typed content-addressed artifacts reopened before grading/settlement | +| Qualification evidence | External evidence root plus deployment-owned expected plan/provider context | Typed attempt and referenced artifacts; full regrade additionally requires reconstructed `CausalQualificationInput`, while settlement restart performs narrower byte/PASS authentication | | Evaluation stack | Referee state | Canonical manifest that may reference hostile proposals | -| Settlement and weight state | Chain-scoped SQLite controller | Transactional single-writer state plus intent/readback journal | +| Settlement and weight state | Chain-scoped SQLite controller | Transactional single-writer state plus projection-linked intent/status journal; live readback vectors are not serialized | | Integrated source | Reviewed source control | Full reviewed commit plus selected-payload and attribution digests | | Engine release | Release publication root | Descriptor-addressed signed tree reopened under an expected public key | @@ -151,8 +175,11 @@ object. Tests mirror the authority boundaries rather than one monolithic integration fixture: -- `test_static.py`, manifest tests, and `test_target_catalog.py` cover hostile +- `test_static.py` and `test_target_catalog.py` cover hostile input and target admission; +- `test_artifact_abi.py`, `test_artifact_device_launch.py`, + `test_artifact_runtime.py`, `test_cuda_cubin.py`, `test_cuda_materialize.py`, and + `test_cute_cubin.py` cover the sealed direct-artifact boundary; - `test_stack_manifest.py`, stack-planning tests, and `test_engine_tree.py` cover canonical composition and integration materialization; - qualification, OCI, and reference-protocol tests cover the B/C/B′/T @@ -165,14 +192,13 @@ When learning a type, search for both its successful construction and its rejection tests. The negative cases usually reveal which fields are security inputs rather than descriptive metadata. -## Legacy and diagnostic paths +## Production authority entry points -Some older simulator and direct-evaluation modules remain useful for tests and -development. In particular, `commit_reveal.py` and the local `commit`, -`reveal`, `ledger`, and `legacy-settle --development-only` commands do not back -the production SQLite controller. Likewise, `eval/_launch.py` and direct -`evaluate`/`bench` flows are not alternate qualification authorities. Start -from the validator loop and qualification runner when auditing production. +Development helpers and test fixtures are not alternate qualification authorities. Audit +production intake from the validator loop and durable intake store, then follow the +registered arena, qualification runner, settlement transition, and weight-publication +journal. A path that does not emit and reopen the typed products for those boundaries +cannot substitute for them. Tests are organized under `tests/` by the same boundaries. Contract tests are often the clearest executable examples because they build exact typed objects diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 71f2551..cb7a707 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -8,6 +8,12 @@ : One reward identity that owns several inseparable slot members and displaces their overlapping singleton identities. +**Artifact provider** +: A validator-registered, data-only policy that fixes one sealed-artifact kind, + binding ABI, build phase, load phase, and reviewed implementation. A bundle + selects a provider by ID; it cannot register callbacks or extend the provider + table. + **B / B′** : Independent complete-engine runs of the frozen incumbent evaluation stack, placed before and after C to measure performance and environmental drift. @@ -22,14 +28,26 @@ remain bound to the crowned payload digest. **Crown** -: A transactional settlement result backed by two independent matching PASS - qualifications. It advances an evaluation stack; it does not ship code. +: A transactional settlement result backed by two matching PASS qualifications whose + seven required authority/attempt/report/selection digests differ. It advances an + evaluation stack; it does not ship code. Digest distinctness does not attest separate + operators, hosts, or infrastructure failure domains. **Discovery lane** : A bounded SGLang source-patch proposal path for improvements that do not yet fit a registered target. Its outcomes are review/promotion decisions, not a permanent whole-engine title. +**Direct-artifact declaration** +: The manifest's closed description of artifact exports, semantic bindings, + specialization, launch parameters, resources, and lifecycle. It contains no + candidate runtime callback. + +**Direct-artifact runtime** +: Validator-owned scheduler-rank code that reopens a sealed publication, admits + its observed device ABI, materializes parameters and resources from the live + slot call, launches it, and emits execution evidence. + **Engine release** : A signed, chain-independent, integrated-only Optima Engine artifact with reproducible runtime builds, exact model receipt, native artifacts, SBOM, @@ -62,12 +80,24 @@ **Reproduction pending** : State after one authoritative PASS. Settlement requires a second independent - matching PASS; the lower reproduced speedup is used. + matching PASS under the seven digest-distinctness checks; the lower reproduced + speedup is used. **Release stack** : Chain-independent `EngineReleaseManifest` containing only integrated contribution references. It has no arena or proposal content. +**Sealed CUBIN publication** +: The content-addressed CUBIN files and canonical index produced by the + registered CuTe provider's hermetic prebuild. It is one provider-specific form + of a sealed direct artifact, not a miner-supplied host launcher or JIT input. + +**Sealed direct artifact** +: A contribution execution form in which the manifest is a closed launch + program and runtime execution uses validator-built, sealed device code. The + provider, compilation profile, publication, observed ABI, and launch plan are + independently bound authorities. + **Slot** : Validator-owned execution ABI inside the pinned engine. Slots define how a component is invoked; targets define how a contribution is identified and diff --git a/docs/reference/manifest-schema.md b/docs/reference/manifest-schema.md index 2652f69..a9b8eb9 100644 --- a/docs/reference/manifest-schema.md +++ b/docs/reference/manifest-schema.md @@ -100,6 +100,8 @@ competitive bundles should declare `[competition]` explicitly. | `base_kernel` | no | Validator-owned base-kernel identifier for an override submission | | `override_point` | no | Typed hole in that base; requires `base_kernel` | | `cuda_sources` | no | Inspectable `.cu`/`.cuh` inputs for a reviewed builder | +| `aot_exports` | no | Ordered sealed direct-artifact exports for a slot with a declarative call ABI | +| `artifact_resources` | no | Validator-allocated storage shared by the row's artifact lifecycle | Unknown operation fields are retained for observation. They do not grant a capability; complete feature resolution can reject a bundle that asks for more @@ -118,6 +120,13 @@ transformation while `entry` names the runtime call. `setup` is a legacy direct framework hook and every current component target forbids it, so new competitive manifests should not use it. +For a row with `aot_exports`, `source` is scanned and its declared compiler +factory is imported only in the isolated prebuild compiler child. The scheduler +does not import that source as a runtime launcher. `entry` remains required by +the outer `optima-op-abi-v0` syntax, but direct execution canonicalizes it to the +validator entry `_optima_direct_artifact`; changing the unused value does not +change direct-execution identity. + ### Variants Several rows may implement the same semantic slot only when every row has a @@ -149,6 +158,122 @@ dtypes = ["bfloat16"] architectures = ["sm103"] ``` +## Sealed direct-artifact exports + +`[[ops.aot_exports]]` is a strict, closed table. It is accepted only for a slot +that has an immutable validator-defined artifact call ABI and only when target +resolution permits the observed provider and rebuild features. + +| Field | Required | Contract | +|---|---:|---| +| `provider` | yes | Registered provider ID; the registry contains `cutlass.cute.cubin.v1` | +| `name` | yes | Export-local canonical identifier | +| `factory` | yes | Identifier called only in the no-egress compiler child | +| `profile_inputs` | yes | Unique list drawn from the provider's compile-profile allowlist | +| `bindings` | yes | Ordered projections of immutable slot resources and declared artifact resources | +| `device_plan` | yes for the registered provider | Complete `optima.device-launch-plan.v1` declaration | +| `role` | no | `init`, `prepare`, `reset`, `run`, or `destroy`; default `run` | +| `plan` | no | Specialization-plan name; default `default` | +| `step` | no | Ordered non-negative step within the plan; default `0` | +| `specializes` | no | Exact-equality predicates over call-ABI resources | +| `prelaunch` | no | Bounded validator operation list; the registered operation is exact `fill` | +| `provider_capability_requirements` | sealed/reopen | Canonical requirements derived from binding projections | +| `specialization_capability_requirements` | sealed/reopen | Canonical requirements derived from specialization sources | + +The provider compile-profile allowlist is +`max_active_clusters.cluster_size_{1,2,4,8,16}`. A factory can receive only the +keys named in its `profile_inputs`; it cannot query a GPU in prebuild. + +TOML row order is not executable authority. The parser canonicalizes exports by +provider, plan, step, and name. Within each plan, steps must be unique and their +step-sorted lifecycle roles must follow `init`, `prepare`, `reset`, `run`, +`destroy`; every step in that plan carries the same specialization predicate. +Plan overlap is legal only as a strict fallback chain. Runtime selects the unique +most-specific exact match rather than using source order. + +### Binding rows + +Each binding row has a `kind` and, except for `aggregate`, a canonical `source` +such as `input.q`, `output.out`, `stream.current`, or a declared +`workspace.*` resource. Available projections are a closed relation to the +source kind: + +| Binding kind | Main projections and options | +|---|---| +| `tensor` | `descriptor`; optional bounded `unsqueeze`, checked `assumed_align`, and `leading_dim` | +| `pointer` | `device_ptr` from tensor storage, `identity` from a pointer, or provider-authorized `native_handle` / `peer_ptr_table` from a group | +| `scalar` | `value`, or tensor/group metadata `shape`, `stride`, `numel`, `rank`, `size`, `storage_offset`; requires an exact scalar `cast` | +| `stream` | `identity` for the validator's captured stream | +| `group` | `identity` for the supplied semantic group | +| `opaque` | `identity` for a validator-held opaque slot resource | +| `aggregate` | Bounded nested `Shape`, `Coord`, `Tile`, `IntTuple`, or `Stride` value with `i32`/`i64` dynamic leaves | + +An axis is required for `shape` and `stride`. A `peer_ptr_table` binding must +name the exact persistent `group_ipc` artifact resource in `peer_resource`. +Bindings cannot name a callback, allocate a new stream or group, or construct an +integer address. + +### Artifact resources + +Each `[[ops.artifact_resources]]` row has exactly these authored fields: + +| Field | Required | Contract | +|---|---:|---| +| `name` | yes | `workspace.*`, `prepared.*`, or `state.*` | +| `dtype` | yes | Registered storage dtype | +| `alignment` | yes | Power of two, at least dtype width and at most 1 MiB | +| `lifetime` | yes | Fixed by the name prefix: `call`, `prepared`, or `engine` | +| `shape` | yes | One to 16 bounded extent tables | +| `scope` | no | `rank_local` by default or persistent `group_ipc` | + +Each shape extent contains `factors` and optional `divisor`. A factor is either +`{ static = N }` or a dynamic table with `source`, `projection`, `upper_bound`, +and `axis` when projecting a tensor dimension. The extent is +`ceil(product(factors) / divisor)`. This is the entire allocation-expression +language. Dynamic sources must be resources in the immutable slot call ABI, not +other artifact buffers. A prepared-lifetime shape may use only resources present +at the validator's exact prepare-allocation boundary. + +Prefix, lifetime, and role are validated together: + +| Prefix | Lifetime | Lifecycle rule | +|---|---|---| +| `workspace.*` | `call` | Cannot be used by `init` or `destroy` | +| `prepared.*` | `prepared` | Must be produced in `prepare` and consumed in `run`; cannot be used by `init` or `reset` | +| `state.*` | `engine` | Persists with the artifact entry and is available to registered lifecycle roles | + +`group_ipc` is invalid for call-local workspace. Manifest ceilings are 32 +resources, 64 GiB per resource, and 128 GiB aggregate; runtimes may impose lower +live limits. These declarations request validator-owned storage, not candidate +allocation authority. + +### Device launch plan + +The registered provider requires `device_plan.schema = +"optima.device-launch-plan.v1"` plus exact `kernels` and `launches` inventories. + +| Inventory | Required contents | +|---|---| +| `kernels` | Sorted unique logical `name` and complete `parameter_sizes` byte vector | +| `launches` | Contiguous `ordinal`, logical `kernel`, `grid`, `block`, optional `cluster`, `shared_mem_bytes`, ordered `parameters`, `stream_binding`, and allowlisted `attributes` | + +Grid, block, cluster, shared memory, and scalar parameter values are checked +expression trees over admitted live bindings. Parameter kinds are `pointer`, +`scalar`, `packed_struct`, `tma_descriptor`, +`cutlass_fast_divmod_i32_v1`, `cute_fast_divmod_i32_v1`, and +`group_handle`. The parameter-size vector of every launch must exactly match the +logical kernel contract. + +The plan schema is bounded to 1,024 logical kernels, 256 launches, and 64 semantic +bindings. Dynamic shared memory is expression-derived; the runtime evaluates it +for each invocation and rejects values above 1 MiB. At runtime, canonical logical +kernel ordinal is matched to the complete driver-observed physical CUBIN inventory +and exact per-ordinal widths. Candidate-provided physical-symbol search is not +part of the ABI. + +See [Sealed direct artifacts](../architecture/direct-artifacts.md) for build, +admission, evidence, identity, and support boundaries. + ## Dependency patches ```toml @@ -186,6 +311,9 @@ A bundle cannot choose: | Duplicate-slot error | Multiple rows without explicit unique variants | Name every variant and make domains disjoint | | Competition error | Unknown target, wrong `slot`/`atomic` mode, legacy `system` title | Choose a registered target from validator output | | Feature-admission error | CUDA, patch, override, setup, or extra capability outside target policy | Remove the feature or choose the registered lane | +| Artifact-provider error | Unknown provider, missing rebuild feature, or non-crownable provider | Use the registered provider only on a target that admits it | +| Artifact-ABI error | Binding source/projection, resource lifecycle, or device parameter widths disagree | Reconstruct the declaration from the slot call ABI and complete CUBIN contract | +| Compile-profile error | Factory requests an undeclared constant or architecture/profile authority differs | Declare only allowlisted inputs and rebuild for the exact arena profile | | Static scan error | Forbidden import/operation or uninspectable tree content | Rewrite the source; scanning is not a sandbox exception list | | Verification error | Callable/signature/output/reference mismatch | Debug the registered tensor and correctness contract | | Qualification failure | Complete engine misses timing, drift, quality, fidelity, or resource gates | Inspect retained arena evidence; do not relabel the outcome | @@ -200,6 +328,8 @@ A bundle cannot choose: contained relative path. - Give repeated slot rows unique variants with non-overlapping domains. - Avoid `setup`; use only registered `prepare` and entry contracts. +- For direct artifacts, declare the complete binding, resource, lifecycle, and + device plan; never package generated CUBIN or a host launcher. - Run `scan`, then the appropriate local or collective `verify` command. - Hash and package the exact verified tree; any later byte change is a new proposal. diff --git a/docs/reference/slots-table.md b/docs/reference/slots-table.md index 62f7dfc..99d036c 100644 --- a/docs/reference/slots-table.md +++ b/docs/reference/slots-table.md @@ -49,9 +49,10 @@ it but may not create a private group or let only some ranks fall back. !!! note "Registered does not always mean installed" `attention.msa_block_score` has a complete slot and verifier contract, but - its decode-side SGLang adapter is deliberately non-installing until the - pinned runtime exposes a stable, auditable chokepoint. Its prefill sibling - has a live adapter. See [Current status](state-of-record.md). + its decode-side SGLang adapter refuses installation unless the pinned + runtime exposes a stable, auditable registered chokepoint. Its prefill + sibling has an installed adapter. See [State of record](state-of-record.md) + for validated coverage. ## Kinds @@ -106,9 +107,9 @@ validator refreshes registered dynamic inputs and poisons outputs between replays before comparing against fresh trusted references. An implementation that cannot establish graph safety is not silently credited -for work the baseline performs. `attention.decode` currently has an -eager gather form; a paged-direct graph-safe form is the intended stronger -boundary. +for work the baseline performs. The `attention.decode` eager gather form does +not establish the paged-direct graph-safe boundary required for graph-qualified +coverage. ### Graph failure examples diff --git a/docs/reference/stack-manifests.md b/docs/reference/stack-manifests.md index 5ef5721..f3faa57 100644 --- a/docs/reference/stack-manifests.md +++ b/docs/reference/stack-manifests.md @@ -23,7 +23,8 @@ arena. It binds: - runtime and base-engine digests; - the arena digest; -- the full target-catalog snapshot and digest; +- the full target-catalog snapshot and digest, including the registered artifact-provider + policy; and - one content-addressed contribution reference per active target. Its entries may be hostile `ProposalContributionRef` values or reviewed @@ -56,6 +57,9 @@ Conceptual shape: - `arena_digest` fixes the hardware/workload/policy comparison domain. - `catalog_snapshot` explains the target policy used at that time, while `catalog_digest` authenticates those exact bytes. +- The catalog snapshot carries both `artifact_provider_registry` and its digest. Provider + kind, ABI, build/load phases, capability vocabulary, and crownability therefore rotate + catalog and stack authority even when the visible target IDs are unchanged. - `entries` names the active contribution for each economic target. Each entry separates the whole artifact digest, selected-payload digest, @@ -150,6 +154,8 @@ selected crowned payload under the current contract. - Parsing is exact-schema and rejects unknown or mistyped structure. - Catalog snapshots travel with their digests; installed policy cannot silently reinterpret retained evidence. +- Artifact-provider policy is part of that catalog snapshot; a later provider registry + cannot reinterpret an older direct-artifact contribution. - Entry keys must equal each reference's target ID. - Target-spec digests must match the bound catalog context. - Active-target composition and displacement are revalidated. @@ -161,6 +167,7 @@ selected crowned payload under the current contract. |---|---| | Unknown field during reopen | The object does not match the exact current schema | | Catalog digest and snapshot disagree | The supplied context is inconsistent or forged | +| Artifact-provider registry or registry digest differs | Native admission/build/load policy belongs to another catalog authority | | Entry key differs from `ref.target_id` | The mapping tries to relabel a contribution | | Target-spec digest differs | Qualification used another semantic contract | | Overlapping active targets | Catalog displacement/composition was not applied canonically | diff --git a/docs/reference/state-of-record.md b/docs/reference/state-of-record.md index c8ab46b..f87ac51 100644 --- a/docs/reference/state-of-record.md +++ b/docs/reference/state-of-record.md @@ -1,253 +1,263 @@ # State of record -This page is the dated evidence ledger for Optima. Architecture pages describe intended contracts; this page records what exists, what has been exercised, and what remains unproven. +This page is the dated implementation and evidence ledger for Optima. Evergreen pages +define contracts and procedures; this page records the source revision, exercised proof +classes, and unresolved limits. A passing test, an empirical qualification, and a +deployable release are different evidence classes. -## Snapshot +## Source snapshot -Snapshot date: **2026-07-13** +Snapshot date: **2026-07-16** -| Item | State | +| Item | Value | |---|---| | Repository | [`latent-to/optima`](https://github.com/latent-to/optima) | -| Branch | `fix/b300-intake-findings` (working branch, three commits ahead of `main`) | -| Commit | `5325468a38d8f12e603de1219de7e0f50ca1a46f` | -| Remote publication | No `origin/*` ref contains this working-branch commit at the snapshot; post-main source paths are therefore not linked as public GitHub files yet | -| Main base | [`990721bc4c6221c4742c1b1827c9aa3048c88628`](https://github.com/latent-to/optima/commit/990721bc4c6221c4742c1b1827c9aa3048c88628), merge of PR #56 | -| Final changes | Fail-closed CuTe DSL compile admission (`804147c4`), scheduler-process-only candidate loading (`3cff4731`), and shared validator-owned DSL-JIT policy extraction (`5325468a`) | -| Production Python | 96 files, 62,773 lines under `optima/` | -| Tests | 92 Python files, 38,539 lines under `tests/` | -| Main-base complete suite | At `990721bc`: 1,702 passed, 3 skipped, 0 failed | -| Current working-tree complete suite | At `5325468a`: 1,724 passed, 18 skipped, 0 failed in 52.45 seconds (`PYENV_VERSION=sn120 python -m pytest -q tests`) | -| Canonical hardening arc | PRs #31–#55, #56's public-key follow-up, then three post-main B300 intake changes on the working branch | - -The line counts are a source snapshot, not a quality metric. Both suite rows -belong to their named commits. The current working-tree suite was rerun while -updating this page; its skipped tests are reported, not silently treated as -passes, and the result is not GPU/B300 empirical evidence. - -## Post-main B300 intake findings - -Three source changes after PR #56 materially refine the current boundary: - -1. `804147c4` - admits CuTe's `cute.compile(<@cute.jit function>, ...)` form under exact - whole-module alias analysis. Only an external, allowlisted `cutlass.cute` - alias bound once is eligible; shadowing, rebinding, local vendoring, other - `.compile` receivers, and `builtins` access fail closed. -2. `3cff4731` - moves candidate loading out of watched-module activation. `seam.activate()` - now arms pass-through adapters but never imports miner code; - `load_candidate_bundle()` runs only at positively identified - `run_scheduler_process` entry. Detokenizer/output-path children therefore - remain candidate-free, and active-member coverage must equal the scheduler - rank count exactly. -3. `5325468a` - extracts tracing-JIT admission into the stdlib-only validator-owned - `DSL_JIT_ENTRYPOINTS` table in `optima/dsl_jit_policy.py`. The shared policy - retains whole-module single-binding and contribution-local shadow checks and - also refuses a string/bytes literal as the first positional compile - argument. CuTe is the only admitted row; Triton needs no explicit - `.compile` carve-out. - -These commits are implemented, focused-tested, and included in the current -working-tree complete suite, but they are not merged into the `main` base named -above and do not erase the outstanding B300 empirical qualification and -release-image proofs below. +| Documentation target | Local `codex/audit-gpu-readiness-20260716` at `f8be20a43708efd830b7b1dae88f00ffaf532ad5` | +| Remote publication | No `origin/*` ref contains the target commit at the snapshot time | +| Main base | [`6d7e92ef9be058821a7cc8cb37aa872cf3af0b68`](https://github.com/latent-to/optima/commit/6d7e92ef9be058821a7cc8cb37aa872cf3af0b68), merge of PR #58 | +| Branch relation | Four local commits ahead of `main` | +| Delta from the prior docs pin | 29 commits; 154 files; +25,690/−5,639 lines | +| Production Python | 108 files, 76,233 lines under `optima/` | +| Tests | 95 Python files, 45,181 lines under `tests/` | +| Complete local suite | 1,951 passed, 19 skipped, 0 failed in 55.55 seconds (`PYENV_VERSION=sn120 python -m pytest -q tests`) | +| SGLang source pin | `0.5.13.post1` in `optima/compat.py` | +| Bittensor raw-reveal storage ABI | `10.3.2` in `optima/chain_canary.py` | + +Line counts describe the pinned tree; they are not quality metrics. The suite result is a +local CPU/non-empirical validation of this checkout. Skips are reported explicitly, and +the result is not GPU qualification, distributed serving, registry reproducibility, or +release evidence. ## Authority order -When sources disagree, use this order: +When sources disagree, apply this order: -1. executable code and schemas at the pinned commit; -2. the [product contract](https://github.com/latent-to/optima/blob/main/docs/PRODUCT_CONTRACT.md) and [slot contract](https://github.com/latent-to/optima/blob/main/docs/SLOT_CONTRACT.md); -3. retained typed evidence and immutable publications; -4. this state page and the [changelog](../history/changelog.md); -5. older plans, audit snapshots, runbooks, and narrative reports. +1. executable code, closed registries, and schemas at the pinned commit; +2. the [product contract](https://github.com/latent-to/optima/blob/main/docs/PRODUCT_CONTRACT.md) + and [slot contract](https://github.com/latent-to/optima/blob/main/docs/SLOT_CONTRACT.md); +3. authenticated retained evidence and immutable publications; +4. this ledger and the [design history](../history/changelog.md); +5. campaign notes, plans, audit snapshots, console output, and older narrative reports. -In particular, [`REFEREE_HARDENING_SPLIT_PLAN.md`](https://github.com/latent-to/optima/blob/main/docs/REFEREE_HARDENING_SPLIT_PLAN.md) and [`REFEREE_HARDENING_AUDIT.md`](https://github.com/latent-to/optima/blob/main/docs/REFEREE_HARDENING_AUDIT.md) are historical snapshots frozen near PR #39. They are not current completion checklists. +The evidence that proves one layer cannot be substituted for another: -## Current product boundary +| Evidence | Establishes | Does not establish | +|---|---|---| +| Unit/focused suite | Implemented invariants under test fixtures | Real GPU behavior or production operations | +| `scan` / `verify` | Static policy and exercised component contract | Model throughput, pristine quality, or settlement | +| One qualification PASS | Registered B/C/B′/T decision under one authority | Crown or release readiness | +| Two independently bound PASSes | Settlement eligibility for the exact contribution/context | Integration or serving authorization | +| Signed release verification | Descriptor, artifact, model, and signature consistency | Reproducible registry build or successful production serving | -The source now implements the product/referee separation: +## Implemented surface -- Optima Engine is represented by `EngineReleaseManifest` and signed release artifacts, without chain runtime dependencies. -- The referee uses `EvaluationStackManifest`, registered arenas, isolated complete-engine execution, pristine reference quality, two-pass reproduction, transactional settlement, and journaled reward projection. -- A crown cannot enter a release until `promote_integrated_contribution()` reopens evidence and produces an approved integration record over reviewed source at a reviewed Git commit. The selected payload remains byte-for-byte bound to its crowned digest; only packaging outside that selected closure and later materialization may be normalized. -- Serving consumes integrated source and signed release identity, never a live proposal URL or chain row. +### Product and referee separation -See [Architecture overview](../architecture/overview.md), [Product model](../architecture/product-model.md), and [Release architecture](../architecture/releases.md). +- `EvaluationStackManifest` represents the complete incumbent and permits hostile proposal + references only inside isolated evaluation. +- Qualification owns B/C/B′ execution, candidate-free pristine T quality, three-way + decisions, and reproduction. Its durable attempt retains aggregate B/C/B′ speed + witnesses plus referenced graph, raw-quality, and T/reference products; richer raw + B/C/B′ session/device frames are validated in-run but are not serialized into the + attempt. +- Settlement updates an arena-scoped evaluation stack with one target transition + transactionally; reward projection and chain publication are separate state machines. +- `EngineReleaseManifest` accepts integrated contributions only. Promotion must reopen + crown evidence and bind reviewed source, selected payload, provenance, security, + compatibility, and tests. +- A serving release has no chain, wallet, intake, settlement, or live proposal dependency. -## Live slot and target surface +### Slots and targets -The executable slot catalog has **11 slots**: +The executable catalog contains 11 slots and one registered atomic target: -1. `activation.silu_and_mul` -2. `norm.rmsnorm` -3. `attention.sdpa` -4. `attention.decode` -5. `attention.msa_block_score` -6. `attention.msa_prefill_block_score` -7. `moe.fused_experts` -8. `moe.fused_experts_reduce` -9. `collective.all_reduce` -10. `collective.ar_residual_rmsnorm` -11. `collective.moe_finalize_ar_rmsnorm` - -The target catalog registers each slot as a singleton and also registers the atomic `collective.moe_epilogue.v1` target over the two MoE epilogue collective members. It defines explicit displacement and the compatible MoE first-applicable composition rule. - -Source: [`slots.py`](https://github.com/latent-to/optima/blob/main/optima/slots.py) and [`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py). - -One distinction is material: `attention.msa_block_score` has a slot/verifier contract, but its decode-side [`sglang_msa.py`](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_msa.py) integration remains a deliberate non-installing stub until a stable pinned-runtime chokepoint exists. Its prefill sibling has a live adapter. - -The source pin is SGLang `0.5.13.post1`. Its static seam canary is green, but -the GPU end-to-end broken-bundle rejection, faithful-bundle pass, and champion -rebaseline are pending at this snapshot. `0.5.12.post1` is the last pin with -complete retained H100 validation. The `compat` command reports, rather than -fails, a pin mismatch, so automation must compare its printed version with -`PINNED_SGLANG` explicitly. - -## Capability matrix - -| Capability | Implemented contract | Validation state at this snapshot | Remaining proof or deployment work | -|---|---|---|---| -| Typed slot ABI | Validator-owned outputs, HP/dequant references, graph inputs, op/block/collective verification | Broad unit and integration coverage; historical real-engine seam/gate exercises exist | Requalify real candidates through the final hardened arena path | -| Target economics | Singleton and atomic identities, overlap/displacement, compatible composition, selected-delta digest | Unit and structural settlement coverage | Calibrate deployed emissions parameters and exercise real crown-backed projection | -| Stack identity | Strict evaluation/release manifests and exact marginal planning | Deterministic identity, transition, cohort, and rollback tests | Operate long-lived real arena stacks and retained rollbacks | -| Engine materialization | Closed source/native closure, deterministic namespaces, reopened tree digests | CPU/focused and RTX lifecycle coverage | Final SM103 native products and joined model/topology proof | -| OCI referee | No-egress workers, read-only roots, bounded mounts/protocol, host timing, device receipts, teardown | Host primitives and B/C/B′ lifecycle exercised on RTX; focused suites green | Full current MiniMax TP4/TP8 B/C/B′/T run on B300/NVLink | -| Pristine T quality | Candidate-free untimed reference protocol, sealed trajectory grading, causal evidence | Joined structural/reference exercises on RTX | Final arena calibration and real candidate quality evidence | -| Finalized intake | Finalized order, hardened HTTPS fetch, re-hash, copy disposition, immutable publication, restart | Live testnet finalized fetch/publication/restart exercised | Sustained production provider, operations, and joined real-candidate crown | -| Arena service | Closed `ArenaServiceRegistry`, five fixed non-crown screens, capacity/retry policy | Implemented and tested through injected in-process registries | No production `ArenaServiceProvider` ships; deployment must supply one | -| Two-pass settlement | Exact reproduction identity, distinct authority/evidence, lower speedup, transactional update | Structural two-`PASS` crown and restart evidence | Real independent-seed current-path crown on final hardware | -| Weight publication | Reopen-before-project, saturating credit, overlap suppression, journal/readback states | Live chain dry-run produced `submitted=false` from a structural crown | No real current-schema empirical crown-backed extrinsic | -| Discovery | Bounded source-patch lane, trusted activation authority, reviewed promotion/bounty model | Joined RTX structural lane exercised; a no-op correctly failed economically | Calibrate bounded pool and operate reviewed promotions | -| Integration promotion | Crown/evidence reopening, review artifacts, source-at-commit verification, immutable attribution | Unit and focused closure coverage | Apply to a real current-path crown and maintain integrated product source | -| Signed release | Deterministic source/wheel, model and native seals, SPDX SBOM, provenance, Ed25519, fail-closed runtime | Release slices and full suite green | No actual final image pair has been built and served end to end | -| Registry/host authorization | Two no-cache builds, Registry-v2 reopening, raw digest equality, signed reproducibility, inspect/seccomp/mount binding | Unit/focused host coverage | Actual registry double-build, digest authorization, TP8 serve, and receipt smoke | -| Production operations | Typed authority surfaces exist | Development/testnet exercises only | Subnet ownership/stake/permits, hosting, monitoring, backups, rollout, incident and key procedures | - -## Hardened-path validation - -The final hardening architecture has meaningful structural validation, but not a final empirical performance crown. - -### What was exercised - -- The exact finalized reveal and publication path was reopened from a live testnet event. -- A structural candidate completed two independent `PASS` authorities, settled into a generation-1 crown, and reopened correctly after restart. -- The structural candidate encoded a synthetic **100-parts-per-million** improvement. It was chosen to prove authority, evidence, and state transitions; it was not an empirical accelerator result. -- A live weight projection ran in dry-run mode and correctly retained `submitted=false`. -- OCI host primitives, materialized engines, native publication, streaming sessions, B/C/B′ lifecycles, pristine T, causal qualification, discovery, settlement, integration, and release components have focused tests and are covered by the current recorded 1,724-passed/18-skipped/0-failed suite. -- Release construction validates external Ed25519 trust, deterministic runtime artifacts, exact native/model/tree bindings, canonical SBOM/provenance, fail-closed runtime environment, registry manifest reopening logic, host inspection policy, and serve-receipt coverage in unit or focused tests. - -### What this does not prove - -The structural proof does not establish a real throughput win, B300 topology behavior, final calibration constants, an actual registry-reproducible image, or successful TP8 serving. It establishes that the authority and persistence machinery can carry an exact two-pass candidate through its state transitions without substituting pre-hardening evidence. - -## Historical performance evidence - -The following are real measurements from earlier Optima paths. They are retained as performance and seam evidence, **not** as crowns produced by the final SQLite/isolated/two-pass/release architecture. - -| Date | Candidate and arena | Result | Historical scope | -|---|---|---|---| -| 2026-07-07 | Shallow fused AR+residual+RMSNorm on MiniMax-M3-NVFP4, TP4, 4×B300, graphs on | 1.044× first run; 1.049× independent prompt-seed reproduction; zero sampled audit violations | Pre-hardening direct referee path | -| 2026-07-07 | Deep MoE finalize+AR+residual+RMSNorm bundle on the same arena | 1.074× first run; 1.071× independent reproduction; zero sampled audit violations | Pre-hardening direct referee path | -| 2026-07-08 | Deep bundle through the then-current public testnet 307 chain loop | 1.072× against a 1.026 bar; score 1.0717; 12,824 sampled calls and zero violations | Historical chain loop before final SQLite intake, isolated authority, and two-pass settlement | -| 2026-07-10 | MSA prefill block-score bundle, two direct TP4 B300 lanes | 1.115× on the ~148.5k-token lane and 1.205× on the ~255k-token lane; 3,148 and 5,804 audited calls, zero violations | Direct `optima evaluate`; no crown or settlement attempted | - -These measurements support the claim that the relevant optimization ideas and seams produced real speedups on the recorded hardware. They do not support the claim that the current hardened evaluator has crowned those artifacts, that current catalog/release identities bind them, or that current emissions may be published from them. - -## Current honest nonclaims - -At this snapshot, Optima does **not** claim any of the following: - -- a real B300 candidate has passed the complete final B/C/B′/T path, independent reproduction, settlement, restart, and crown-backed weight publication; -- the signed release path has built two real registry images with the same raw manifest digest and then served one through the final host/runtime path; -- a final release has completed TP8 serving and all-rank active/fired/completed receipt verification; -- a production `ArenaServiceProvider` is included; -- SM103/CuTe compilation, NVLink/P2P/custom collective behavior, topology calibration, and TP4 half/role swaps are proven for the final path; -- emissions half-life, discovery lifetime/pool, margins, noise envelopes, or refresh policy are production-calibrated; -- mainnet operations, hosted artifact durability, monitoring, backups, rollout, and signing-key procedures are complete. - -## B300 validation debt - -The remaining hardware-specific proof is concentrated rather than architectural. A final B300 campaign must establish: - -1. the exact SM103/CuTe and native-artifact build/load identities; -2. NVLink, P2P, NCCL, custom collective, and fused-epilogue behavior under the registered topology; -3. graphs-on verification and adversarial graph replay for real candidates; -4. calibrated B/B′ drift, speed margin, T quality, hidden-task, and retry policies; -5. concurrent TP4 halves where registered, followed by a role/half swap; -6. one real candidate through first `PASS`, independent reproduction, settlement, crown restart, and crown-backed weights; -7. integration of that crowned delta into reviewed source; -8. an actual signed release, reproducible registry pair, authorized image, TP8 serve, and complete seam receipts. - -None of these items requires changing the product boundary. A failure may still expose an implementation defect, but the work is validation of the current architecture rather than permission to reuse a legacy crown. - -## Operational caveats - -### Executor manager single-writer assumption - -The OCI executor's quiescence lock aliases one manager instance's `transaction_lock` ([`oci_backend.py`](https://github.com/latent-to/optima/blob/main/optima/eval/oci_backend.py)). Safety therefore assumes one authoritative executor manager per protected device/resource pool in a process. Constructing independent managers over the same pool would not create a shared lock and is outside the documented operating invariant. - -### Evidence-record range - -Tracked referee evidence records currently cover PRs #40 through #54. No `pr-0055` or `pr-0056` record exists, and [`check_referee_evidence.py`](https://github.com/latent-to/optima/blob/main/scripts/check_referee_evidence.py) freezes the expected range below 55. Extending the registry requires an explicit schema/range update rather than placing unchecked files beside it. - -### Legacy code - -Legacy developer and simulator code remains in the tree, including [`eval/_launch.py`](https://github.com/latent-to/optima/blob/main/optima/eval/_launch.py), [`eval/capability.py`](https://github.com/latent-to/optima/blob/main/optima/eval/capability.py), [`eval/throughput_kl.py`](https://github.com/latent-to/optima/blob/main/optima/eval/throughput_kl.py), and [`commit_reveal.py`](https://github.com/latent-to/optima/blob/main/optima/commit_reveal.py). Production authority roots are separated from these paths, but they have not been deleted. - -The local simulator commands are `commit`, `reveal`, `ledger`, and `legacy-settle --development-only`. They are not alternate interfaces to production settlement. - -## Public command surface - -### Miner and developer - -```text -slots -scan -verify -evaluate -bench -hash -compat -chain-compat -chain-register -chain-package -chain-submit -chain-status -``` - -`verify`, `evaluate`, and `bench` are diagnostic. Production crown authority begins only after finalized chain intake and registered arena qualification. - -### Validator and rewards +| Kind | Registered identifiers | +|---|---| +| Op | `activation.silu_and_mul`, `norm.rmsnorm` | +| Block | `attention.sdpa`, `attention.decode`, `attention.msa_block_score`, `attention.msa_prefill_block_score`, `moe.fused_experts` | +| Collective | `moe.fused_experts_reduce`, `collective.all_reduce`, `collective.ar_residual_rmsnorm`, `collective.moe_finalize_ar_rmsnorm` | +| Atomic | `collective.moe_epilogue.v1` over the two MoE epilogue collective members | + +Every slot has a singleton target. The atomic target declares displacement and compatible +composition explicitly. `attention.msa_block_score` has a verifier contract but no +installing decode-side SGLang adapter; its prefill sibling has the live adapter. Runtime +MSA `topk` controls downstream block selection, while the slot correctness `top_k` is only +the verifier's overlap width. + +Sources: [`slots.py`](https://github.com/latent-to/optima/blob/main/optima/slots.py), +[`target_catalog.py`](https://github.com/latent-to/optima/blob/main/optima/target_catalog.py), +and [`seams.py`](https://github.com/latent-to/optima/blob/main/optima/seams.py). + +### Sealed direct artifacts + +The closed artifact-provider registry contains one authoritative, crownable provider: +`cutlass.cute.cubin.v1`. + +Its contract is: + +- candidate CuTe factory code runs only in a GPU-hidden, no-network hermetic prebuild + child; +- the child must emit exactly one CUDA ELF CUBIN—never PTX, a host object, shared library, + Python launcher, or runtime JIT input; +- the manifest carries declarative bindings, specialization, launch, resource, and + lifecycle data rather than candidate host callbacks; +- the host seals build, tree, compiler, compile-profile, architecture, CUBIN, kernel ABI, + and launch-plan identities; +- each rank binds logical exports by physical kernel ordinal only after selecting its CUDA + device and initializing process groups; and +- launch materialization, pointers, scalars, packed fields, TMA descriptors, FastDivmod + values, streams, generated storage, cleanup, and evidence remain validator-owned. + +All registered targets allow the provider feature and all 11 slots expose declarative +call ABIs. That is schema coverage, not empirical proof of every slot/provider pair. +Direct-artifact collective declarations and group/peer resources are schema vocabulary, +not an executable standard path: `build_cute_cubin` supplies no group +capability/handle resolvers, so such plans fail closed. The fused-epilogue collective +evidence uses the separate hardened native-build/overlay lane. + +See [Sealed direct artifacts](../architecture/direct-artifacts.md). + +### Public CLI + +The live commands are: ```text -chain-validate --intake-only -chain-validate --arena-id # requires an injected ArenaServiceRegistry -set-weights # requires explicit economic policy arguments +slots compat chain-compat scan verify +chain-package chain-submit chain-status chain-register chain-validate +set-weights model-provision release-verify release-context ``` -The standalone CLI cannot synthesize a production arena provider. `set-weights` separates dry-run/reconciliation from genuine publication and never treats an SDK return as confirmation. +The local loop is `scan` plus `verify`. Complete-engine speed and quality authority begins +with a deployment-injected arena service. The direct evaluator, benchmark command, local +JSON ledger, and standalone hash command are absent from this source tree. -### Release operations +## Validation matrix -```text -model-provision -release-verify -release-context -``` - -Release creation and signing are typed build APIs, not a `release-create` CLI command. +| Area | Implemented contract | Retained evidence | Limit at this snapshot | +|---|---|---|---| +| Bundle and target intake | Canonical content identity, strict extraction, recursive scan, target resolution, immutable publication | Full suite; live testnet intake/restart evidence | Archive PAX/GNU metadata expansion is not included in all resource accounting | +| Component verification | Typed outputs, trusted references, variant routing, graph replay, distributed collective checks | Full suite and earlier GPU controls | `verify` is diagnostic; candidate code and its local comparator share a worker process | +| Direct CUBIN provider | Closed provider, exact compile profile, hermetic child, sealed CUBIN, ordinal ABI, validator-owned launch | Ledger-attested B300 TP4 blockscore load/invoke/complete; retained B200 joined run | B300 receipts were lost with the pod; arbitrary direct collectives are not empirically proven | +| OCI qualification | No-egress prebuild/runtime/reference sessions, B/C/B′/T roles, host timing, device receipts, teardown | Retained B200 primary and reproduction PASSes | No fresh run binds the empirical result to current HEAD; full regrade also requires a reconstructed private `CausalQualificationInput` not embedded in the attempt/SQLite | +| Quality authority | Candidate-free pristine T, sealed trajectories, frozen calibration, retained raw products | B200 joined qualification evidence | In-engine slot audit is not a universal crown gate; minimum external-quality policy is arena-specific | +| Settlement | Exact pair identity, distinct authorities/evidence, lower speedup, transactional generation update | B200 generation-1 crown and restart proof | Held reservations require audited `release_hold` or minimum-age `expire`; no typed arena-retirement API exists | +| Chain intake | Finalized ordering, bounded reveal decoding, strict storage/event equality, retained-only internal processing | Public testnet 307 intake and restart evidence | Retained-only mode is programmatic; no public CLI flag exists | +| Weight publication | Reopen-before-project, intent journal, failed-extrinsic truthfulness, pending-intent resume across heads | Real subnet-307 projection submitted | Final timelock reveal/readback is not independently confirmed; metagraph observations are not block-atomic | +| Arena deployment | Closed `ArenaServiceRegistry`, fixed non-crown screens, capacity and retry policy | In-process and campaign-specific providers | The repository ships no production `ArenaServiceProvider` | +| Integration | Crown reopening and exact source/payload/review binding | Unit and focused tests | No retained B200 crown has been promoted into maintained integrated source | +| Release | Deterministic runtime artifacts, model/native seals, SBOM, provenance, Ed25519, registry and serve types | Unit/focused tests only | Serving-wheel closure, provider-specific native admission, and end-to-end registry/serve authority remain incomplete | +| Operations | Typed state and recovery surfaces | Development and testnet exercises | Mainnet ownership, stake/permits, hosting, backup, monitoring, rollout, and key custody remain deployment work | + +## Empirical qualification evidence + +The strongest retained performance authority is the joined blockscore qualification on a +TP4 arena hosted by an 8×B200 system: + +| Field | Primary | Reproduction | +|---|---:|---:| +| Charged-basis speedup | 1.0561× | 1.0487× | +| Timed-section-only diagnostic | 1.0932× | 1.0866× | +| Qualification decision | PASS | PASS | + +Settlement accepted the pair after all seven required authority/evidence digests differed, +used the lower value, **1.0487×**, and committed a generation-1 crown. Restart reopening +passed. These checks prove distinct bound qualification products; they do not by themselves +prove separate operators, hosts, or infrastructure failure domains. The charged basis +includes conditioning and timed work; the timed-section result and the standalone +19.6–22.4% serving bracket use different denominators and are not interchangeable scores. + +Evidence boundaries: + +- The run used an SM100/B200 worker image from an earlier branch, before the evaluator + trim and strict-validation branch. The relevant production pipeline was preserved, but + current `f8be20a4` has not been rerun on that GPU arena. +- Its SGLang source build was `0.0.0.dev1+g56e290315`, not the repository constant + `0.5.13.post1`. `compat` reports this mismatch without failing, so the run does not + validate the source pin as a consensus boundary. +- A real projection with digest prefix `fb20bc7b` was submitted on subnet 307. The latest + retained record still marked the timelock reveal pending; final chain readback is not + claimed. +- Earlier fused-epilogue and MSA measurements remain useful mechanism evidence in the + [MiniMax-M3 case study](../validator-guide/minimax-m3-case-study.md) and + [design history](../history/changelog.md), but they do not replace this authority or + become current-schema crowns retroactively. + +## Current checkout hardening and release status + +The strict-validation commits centralize repeated validation mechanisms in +[`optima/_strict.py`](https://github.com/latent-to/optima/blob/7c171d3029a97341c255f859a0fc38a23cffada9/optima/_strict.py). +Module-specific exception types and policy bounds remain local; shared helpers enforce +canonical digests, exact integers, identifiers, exact fields, duplicate-key rejection, +CUDA driver integers, and environment flags. Most identity authorities now reject the +all-zero digest. Identity computation itself is unchanged. + +The checked-out branch is **not release-ready**. The serving-wheel allowlist omits the +direct-artifact runtime module closure, including provider, ABI, device-launch/runtime, +CUDA, and CuTe modules. Commit `f8be20a4` added `optima._strict` and a clean extracted-wheel +smoke for `_strict`, the seam, bootstrap, the SGLang plugin, and `release_runtime`; that +narrower regression passes. Clean-wheel `import optima.manifest` still fails first on +missing `optima.artifact_provider`, and other release-path imports fail on omitted modules +such as `optima.cuda_cubin`. The 1,951-test suite does not exercise the complete +manifest-reachable installed-wheel import graph. + +Direct-CUBIN serving has an independent fail-closed blocker: `release_runtime.py` does +not propagate the signed native build's `OPTIMA_CUTE_COMPILE_PROFILE_DIGEST` into the +engine process, while the CUBIN runtime requires that exact profile identity. + +Release preparation also treats native output generically: it reopens a caller-supplied +`NativeArtifactPublication` inventory and binds its `NativeBuildSpec`; it does not run the +provider prebuild or provider-specifically reopen the CuTe index and compile profile. A +release pipeline must retain and verify those provider-specific authorities before the +generic release identity can be accepted. + +Release authorization also requires closure of the open review surface around: + +- effective SGLang argv authority, including abbreviations, duplicate/loader flags, and + management interfaces; +- binding the signed container-reproducibility statement to the actual builder output, + not only a later mutable tag readback; +- authenticated or disabled management routes when serving uses host networking; +- release/session binding for serving receipts and complete archive/model/build TOCTOU + accounting; and +- production key custody, rotation, rollback, registry retention, and deployment policy. + +No final deterministic registry pair, authorized image, TP8 serving run, or complete +all-rank serving receipt set is claimed for current HEAD. + +## Operational invariants and open controls + +- One authoritative executor manager must own each protected device/resource pool. Two + independent managers do not share the quiescence lock. +- Settlement restart authenticates retained attempt bytes and stored PASS dispositions; it + is not a full causal regrade. Preserve the private provider/plan authorities needed to + reconstruct the exact `CausalQualificationInput` if regrade is an operational promise. +- Queue-age admission backpressure is not automatic disposal. An operator must explicitly + release a hold with an audited reason or use the typed expiry transition after its + minimum age; stale non-terminal rows can otherwise block overlapping settlement. +- Generation-zero or orphan arena stacks can block global projection. The repository has + no typed retirement API; direct database deletion is not a supported operating path. +- Weight reconciliation refreshes the metagraph and resumes an exact retained pending + intent across later heads. Its observations are not an atomic chain snapshot, and its + journal deadline is not timelock-reveal-aware. +- FlashInfer dependency overlays build each logical architecture in a separate hermetic + child because architecture selection is import-time state. Runtime installs only the + row for the active device. During autotuning, both the deep producer and fused epilogue + consumer remain stock so a candidate cannot influence tactic selection. +- The DSL-JIT carve-out admits only the registered CuTe form under closed binding and + contribution-local reachability rules. It is defense in depth inside OCI isolation, + not a proof that arbitrary Python or native code is safe. ## Source map -- [`docs/PRODUCT_CONTRACT.md`](https://github.com/latent-to/optima/blob/main/docs/PRODUCT_CONTRACT.md) -- [`docs/SLOT_CONTRACT.md`](https://github.com/latent-to/optima/blob/main/docs/SLOT_CONTRACT.md) -- [`docs/EMISSIONS_POLICY.md`](https://github.com/latent-to/optima/blob/main/docs/EMISSIONS_POLICY.md) -- [`chain/intake.py`](https://github.com/latent-to/optima/blob/main/optima/chain/intake.py) -- [`eval/qualification_runner.py`](https://github.com/latent-to/optima/blob/main/optima/eval/qualification_runner.py) -- [`settlement.py`](https://github.com/latent-to/optima/blob/main/optima/settlement.py) -- [`release.py`](https://github.com/latent-to/optima/blob/main/optima/release.py) -- [`release_host.py`](https://github.com/latent-to/optima/blob/main/optima/release_host.py) -- [`release_runtime.py`](https://github.com/latent-to/optima/blob/main/optima/release_runtime.py) +- [Artifact provider policy](https://github.com/latent-to/optima/blob/main/optima/artifact_provider.py) +- [Artifact call ABI](https://github.com/latent-to/optima/blob/main/optima/artifact_abi.py) +- [CUDA device launch authority](https://github.com/latent-to/optima/blob/main/optima/artifact_device_launch.py) +- [CuTe CUBIN publication/runtime](https://github.com/latent-to/optima/blob/main/optima/cute_cubin.py) +- [OCI prebuild](https://github.com/latent-to/optima/blob/main/optima/eval/oci_prebuild.py) +- [Qualification runner](https://github.com/latent-to/optima/blob/main/optima/eval/qualification_runner.py) +- [Finalized intake](https://github.com/latent-to/optima/blob/main/optima/chain/intake.py) +- [Settlement](https://github.com/latent-to/optima/blob/main/optima/settlement.py) +- [Weight reconciliation](https://github.com/latent-to/optima/blob/main/optima/chain/weights.py) +- [Release construction](https://github.com/latent-to/optima/blob/main/optima/release.py) +- [Release host authority](https://github.com/latent-to/optima/blob/main/optima/release_host.py) +- [Release runtime](https://github.com/latent-to/optima/blob/main/optima/release_runtime.py) diff --git a/docs/reference/target-catalog.md b/docs/reference/target-catalog.md index 5cd0b37..a552043 100644 --- a/docs/reference/target-catalog.md +++ b/docs/reference/target-catalog.md @@ -27,7 +27,7 @@ The resulting target-spec digest travels with proposals, stack entries, qualification evidence, integration records, and releases. A target ID without that digest is insufficient to reopen historical authority. -## Current targets +## Registered targets The `target-catalog.v1` policy contains the 11 singleton slot targets plus one atomic target: @@ -54,10 +54,10 @@ active alongside the atomic target. This prevents one semantic change from creating duplicate permanent reward titles. Catalog registration defines identity and admission; it does not by itself -prove that a serving seam is live. In particular, the -`attention.msa_block_score` decode-side integration is currently a deliberate -non-installing stub pending a stable pinned-runtime chokepoint. See -[Current status](state-of-record.md). +prove that a serving seam is installed. The `attention.msa_block_score` +decode-side integration is non-installing unless the pinned runtime exposes a +stable registered chokepoint. See [State of record](state-of-record.md) for +validated adapter coverage. ## Resolution diff --git a/docs/security/anti-copy.md b/docs/security/anti-copy.md index 898e5d6..92c6164 100644 --- a/docs/security/anti-copy.md +++ b/docs/security/anti-copy.md @@ -88,9 +88,8 @@ They can arise from common utility code, simple contracts, or independent implem The comparison records them as advisory context for review. Structural normalization blanks Python identifiers and constants while retaining -operations and attribute structure. There is no equivalent semantic CUDA parser in the -current implementation. Renaming CUDA identifiers and changing constants can evade that -advisory layer. +operations and attribute structure. The automatic signal has no equivalent semantic CUDA +parser. Renaming CUDA identifiers and changing constants can evade that advisory layer. ## Discovery diff --git a/docs/security/evidence.md b/docs/security/evidence.md index ee6ff0a..b99e4f5 100644 --- a/docs/security/evidence.md +++ b/docs/security/evidence.md @@ -1,15 +1,20 @@ # Evidence and replay -Optima retains raw, content-addressed evidence so a qualification decision can be -reopened and regraded. A summary score, log excerpt, or database flag is not sufficient -authority by itself. - -The database and evidence store play complementary roles. SQLite says **which transition -was authorized, in what order, and which evidence addresses belong to it**. The evidence -store supplies **the authenticated bytes needed to recompute the decision**. Backing up -only one leaves either an unindexed pile of artifacts or an unreplayable state machine. - -## Two durable layers +Optima retains typed, content-addressed qualification attempts and referenced evidence. A +summary score, log excerpt, or database flag is not sufficient authority by itself. The +persisted attempt is also not a self-contained full-regrade package: causal regrade +requires the caller to supply the exact reconstructed `CausalQualificationInput`, including +private plan/provider authorities and graph/calibration references not embedded in SQLite +or `CohortQualificationAttempt`. + +SQLite, the evidence store, and deployment-owned regrade context play complementary roles. +SQLite says **which transition was authorized and in what order**. The evidence store +authenticates **the attempt and referenced artifacts that were published**. The external +context supplies **the expected causal plan, requirements, and artifact references needed +by the full grader**. Backing up only the first two preserves settlement byte authority but +does not guarantee that the complete qualification decision can be regraded. + +## Durable state and regrade authority ### SQLite authority @@ -44,9 +49,21 @@ bytes at the same address fail. The default artifact limit is 64 MiB, with a hard 1 GiB ceiling. Individual evidence producers may impose tighter bounds. +### Regrade context + +`reopen_causal_qualification(..., expected=...)` requires a reconstructed +`CausalQualificationInput`. The expected object supplies the full prepared plan, candidate +authorities, graph evidence references and requirements, calibration artifact/reference +context, runtime resource policy, T/reference authority, and commitment. The persisted +`QualificationAuthorityManifest` carries identity fields and commitments, not this full +object. Production retention must therefore preserve enough reviewed private deployment +state to reconstruct the exact expected input; a digest without the typed object is not a +regrade recipe. + ## Evidence chain -Authoritative qualification relates these products: +Authoritative qualification relates these products. The table is a conceptual authority +chain, not a claim that every row is embedded in the attempt or SQLite: | Stage | Representative retained identity/evidence | |---|---| @@ -54,15 +71,15 @@ Authoritative qualification relates these products: | Stack construction | Target catalog, contribution ref, incumbent/candidate manifests, materialized trees, marginal arm/cohort plans | | Arena | Service manifest, capacity decision, ordered screen-stage evidence and receipt | | Launch | Runtime preflight, model mount, native build/publication, hardware/resource/seccomp identity | -| Execution | B/C/B′ raw session frames, prompt batches, token counts, timings, device-state and cleanup receipts | +| Execution | Durable `SpeedWitness` with three aggregate B/C/B′ charged-rate rows; richer session frames and per-arm device/cleanup observations are validated in-run but are not serialized into the attempt | | Graph | Member, variant, shape, capture, and replay observations | | Selection | Pre-execution commitment, post-commit entropy, secret reveal, selected prompts, sealed trajectory digest | -| Reference | Candidate-free T request/transcript and teacher/hidden-task evidence | +| Reference | Candidate-free T session/request/frame identities plus the referenced raw teacher/rollout/hidden-task quality artifact; raw request/response transcript bytes are not embedded in the attempt | | Calibration | Raw control evidence, context, frozen thresholds and metric policy | | Qualification | Regraded candidate report and complete cohort attempt | | Reproduction | Distinct authority, attempt, report, and selection evidence over the same economic identity | | Settlement | Paired candidate, evidence references, hash-chained event plan, atomic stack transition | -| Weights | Exact global projection, intent/pending/held/confirmed records, live readback | +| Weights | Exact retained projection plus journal status/blocks/`last_update`/reason; live vectors are compared during reconciliation but pre/post readback vectors and post-read metagraph identity are not serialized | | Release | Integration records, model receipt, source/wheel, native inventory, SBOM, provenance, descriptor signature, OCI attestation, serve receipts | Not every artifact is public. Selection secrets and hidden tasks remain in private @@ -72,24 +89,29 @@ the secret bytes. ```mermaid flowchart LR DB["SQLite state transition\nordered refs + statuses"] - ES["Evidence store\ndomain + schema + bytes"] - RG["Regrader\nfrozen policy + raw evidence"] + ES["Evidence store\nattempt + referenced bytes"] + PA["Private expected input\nplan + provider + graph/calibration refs"] + RG["Causal regrader\nexpected input + authenticated products"] OUT["Reconstructed verdict\nPASS / FAIL / NO_DECISION"] - DB -->|"artifact refs + exact context"| RG + DB -->|"recorded identities + disposition"| RG ES -->|"authenticated artifacts"| RG + PA -->|"reconstructed CausalQualificationInput"| RG RG --> OUT OUT -. "must match retained transition" .-> DB ``` ## Reopening and regrading -A valid audit proceeds from identities, not from a desired verdict: +A full evidence regrade is available only when the exact expected +`CausalQualificationInput` can be reconstructed. It proceeds from identities, not from a +desired verdict: 1. Reopen the chain-scoped reservation and immutable publication. 2. Reopen the exact target catalog, stack/tree, launch, model, and native identities. 3. Authenticate each evidence artifact by domain, schema, size, and digest. -4. Recompute charged execution rates from raw session batches. +4. Validate the three aggregate `ChargedExecutionRate` rows, recompute their tokens/second + values from retained counts and intervals, and regrade speed under frozen calibration. 5. Regrade graph observations against the frozen requirement. 6. Reconstruct selection and verify post-commit entropy and trajectory binding. 7. Verify T is candidate-free and covers the selected prompts/tasks. @@ -97,28 +119,55 @@ A valid audit proceeds from identities, not from a desired verdict: 9. Reconstruct the qualification decision and reproduction identity. 10. Reopen settlement and reward claims before projecting weights. -Missing, changed, ambiguous, or context-mismatched evidence yields a hold or -`NO_DECISION`; it must not be patched over with an operator assertion. +Missing, changed, ambiguous, or context-mismatched evidence prevents a valid full regrade; +it must not be patched over with an operator assertion. The live qualification path maps +its recognized authority failures to `NO_DECISION`; a later audit must describe missing +external regrade context as an unavailable proof, not fabricate a new verdict. + +Settlement recovery is narrower. It reopens and authenticates the retained attempt bytes, +checks the stored PASS disposition and seven digest-distinctness fields, and binds the +settlement transition. It does not call the full causal grader or recover graph/calibration +references from the attempt. “Reopened for settlement” and “fully regraded” are therefore +different evidence claims. + +### Sampled seam audit is not universal authority + +[`optima/audit.py`](https://github.com/latent-to/optima/blob/main/optima/audit.py) +can emit sampled comparison receipts for supported live dispatch seams when its runtime +flags are explicitly armed. The isolated qualification worker currently clears those +flags, and qualification does not consume sampled audit receipts as a required input. +They are therefore optional diagnostics, not a universal cross-slot correctness gate or +settlement authority. + +Required authority still comes from the registered verifier and graph checks, canonical +execution receipts, B/C/B′ measurement, and candidate-free T policy. The absence of a +sampled audit receipt must not be rewritten as either a pass or a failure unless a future +typed policy explicitly binds, retains, and grades that evidence. ### Example: reopening a disputed speed pass A reviewer starts with the settlement candidate, not with a dashboard's reported speedup. They reopen both qualification attempt references and verify that their -reproduction identities match but their authorities/evidence roots are distinct. For -each attempt they then: - -1. recover the exact B/C/B′ raw batches and host timings; -2. verify token counts, charged intervals, role order, device-state, and cleanup receipts; -3. recompute B/B′ drift and candidate throughput under the frozen calibration; +reproduction identities match while the seven required authority, attempt, report, +commitment, and selection digests differ. The references may share one +content-addressed store root. For each attempt they then: + +1. reopen each report's `SpeedWitness` and verify its B/C/B′ launch and distinct session + identities; +2. verify retained conditioning/timed/charged token counts, intervals, sums, and the witness + projection digest; +3. recompute the three aggregate rates, B/B′ drift, and candidate speed decision under the + frozen calibration; 4. reopen selection commitments and the sealed C trajectory; 5. verify the separate T lifetime is candidate-free and regrade quality; 6. reconstruct the attempt verdict; and 7. verify settlement used the lower accepted speedup and the exact live target transition. -If the dashboard rounded both attempts to “1.04×” but raw evidence reconstructs different -accepted values, the exact values and conservative settlement rule govern. If one raw -session artifact is gone, the reviewer cannot fill the gap from logs or the other pass; -the economic claim must be held according to policy. +If the dashboard rounded both attempts to “1.04×” but the retained aggregate witnesses +regrade to different accepted values, those exact values and the conservative settlement +rule govern. The persisted attempt cannot answer a dispute that requires raw batch frames, +per-arm device samples, or lifecycle receipts; logs must not be presented as if the schema +had authenticated those absent products. ## Failure semantics @@ -126,7 +175,7 @@ the economic claim must be held according to policy. |---|---| | Artifact digest, length, domain, media type, or schema differs | Authentication failure; do not consume the bytes | | Artifact exists but belongs to another arena/stack/attempt context | Context mismatch; it cannot repair this authority | -| Raw execution evidence is incomplete | No attributable reconstructed verdict; hold or `NO_DECISION` | +| Required attempt/witness or referenced graph/quality evidence is incomplete | No attributable regraded verdict; hold or `NO_DECISION` | | Regraded result differs from stored verdict | Preserve both products, stop downstream transition, investigate policy/code/state integrity | | Two passes reuse authority or evidence | Reproduction requirement is unsatisfied | | Active claim's evidence was deleted | Hold reward projection; do not treat deletion as retirement | @@ -149,14 +198,25 @@ disaster recovery. A production plan should define: Deleting evidence for an active crown is not harmless garbage collection: economics will hold the projection when the claim can no longer reopen. +Retention clocks are type-specific. Settlement lease expiry can recycle a lease, and a +discovery claim may have a policy lifetime. Intake reservations do not expire +automatically: an operator may call the typed `expire` transition only after the configured +minimum age, or release a held reservation with an audited reason. Neither operation is a +typed retirement of an entire arena. Until one of those evidence-preserving row +transitions succeeds, a durable hold remains authoritative; backup or cleanup procedures +must not treat age alone as retirement. + ### Restore drill A useful restore test uses a copy of the SQLite database and evidence root on a clean -owner-controlled path, then exercises the same reopen/regrade calls used by settlement and -weight projection. Verify not only digests, but also the filesystem assumptions enforced -by the evidence store: absolute root, owner-only directories, regular single-linked files, -read-only modes, and stable size/digest during access. A byte-perfect backup restored with -unsafe ownership or hard links should fail until the storage boundary is repaired. +owner-controlled path. Test two paths separately: settlement's attempt-byte/PASS reopen, +and full causal regrade using a separately restored exact `CausalQualificationInput`. +Weight projection reopening is a third state path. Verify not only digests, but also the +filesystem assumptions enforced by the evidence store: absolute root, owner-only +directories, regular single-linked files, read-only modes, and stable size/digest during +access. A byte-perfect backup restored with unsafe ownership or hard links should fail; +an SQLite/evidence restore without the private expected input must not be reported as a +successful full regrade. ## Privacy and observability @@ -170,6 +230,12 @@ instead of dumping candidate source, hidden work, or model data. - Content addressing proves byte integrity, not semantic correctness. - Regrading reproduces the implemented policy; it does not prove the policy was well chosen. +- Weight projection evidence binds the desired vector and projection-time metagraph/state + identities. The publication journal records status and chronology metadata, not the + pre/post readback vectors or post-read metagraph identity; it neither authenticates those + live observations nor turns separate SDK queries into a block-atomic snapshot. +- Optional sampled in-engine audit receipts are not a universal qualification or + settlement authority. - Local retention does not provide cross-validator consensus or public availability. - A database administrator or host root can still destroy or replace the entire authority domain; backups, access control, and independent audit remain operational requirements. diff --git a/docs/security/isolation.md b/docs/security/isolation.md index 108e72a..04287eb 100644 --- a/docs/security/isolation.md +++ b/docs/security/isolation.md @@ -60,9 +60,11 @@ The disposable prebuild container receives: - one lease-scoped writable staging mount. It receives no network, GPU, model, home directory, wallet, Docker socket, or ambient host -mount. Candidate files are parsed as data, and only validator-registered rebuild patchers -may run in the build phase. The host never imports candidate Python or loads the produced -native object. +mount. The top-level prebuild parses candidate files as data and invokes only +validator-registered rebuild patchers. A registered sealed-direct-artifact patcher may +start a further bounded compiler child, import the declared candidate factory there, and +call it under the closed compile-request/profile ABI. The host never imports candidate +Python or loads the produced native object. Prebuild container controls include: @@ -146,11 +148,13 @@ fails as infrastructure authority; it must not become a candidate `PASS` or `FAI ## Static scanning and Python subprocesses -Source scanning remains useful defense in depth. It rejects known-dangerous syntax and -recursively covers declared source, CUDA sources, patches, and vendored tree content. -But scanning cannot prove arbitrary code safe, especially generated native code. +Source scanning remains useful defense in depth. It applies the Python AST policy to every +declared and vendored `.py` file. Manifest-declared CUDA sources and dependency patches are +recognized as separate reviewed-build tiers; they are not Python-AST scanned. The tree +guard rejects undeclared executable files, binary artifacts, and symlinks. None of these +checks proves arbitrary code safe, especially generated native code. -The current scanner admits the required CuTe DSL form `cute.compile(...)` only +The scanner admits the required CuTe DSL form `cute.compile(...)` only when the receiver is bound exactly once by a plain absolute alias of the allowlisted external `cutlass.cute` module and the first positional argument is not a string/bytes literal. Rebinding, shadowing, a second import, or a @@ -166,9 +170,9 @@ wrapped scheduler-process entry. Detokenizer/output-path and other manager children never import candidate code or write an `active` receipt; exact active-member coverage must equal the expected scheduler rank count. -`verify` uses fresh spawned workers so the CLI process does not import normal candidate -entry code. `evaluate` and `bench` load candidate code in model workers and can request a -diagnostic no-egress network namespace. These are development properties, not a substitute +Component verification uses fresh spawned workers so the CLI process does not import +normal candidate entry code. Developer model launches may load candidate code in model +workers and may add a diagnostic no-egress namespace. Those properties do not substitute for the production OCI, mount, device, evidence, and cleanup authority. ## Serving releases are a different boundary @@ -222,4 +226,4 @@ See [Threat model](threat-model.md) and [Evidence and replay](evidence.md). - [OCI process leases](https://github.com/latent-to/optima/blob/main/optima/eval/oci_process.py) - [OCI runtime backend](https://github.com/latent-to/optima/blob/main/optima/eval/oci_backend.py) - [Reference session](https://github.com/latent-to/optima/blob/main/optima/eval/oci_reference_session.py) -- Scheduler-only candidate load gate: post-main working-tree path `optima/integrations/sglang_scheduler_gate.py` +- [Scheduler-only candidate load gate](https://github.com/latent-to/optima/blob/main/optima/integrations/sglang_scheduler_gate.py) diff --git a/docs/security/threat-model.md b/docs/security/threat-model.md index 09efca7..22f7a5b 100644 --- a/docs/security/threat-model.md +++ b/docs/security/threat-model.md @@ -130,10 +130,10 @@ host networking; it is not the hostile candidate container. ## Non-authoritative development paths -`verify`, `evaluate`, and `bench` are diagnostics. Their spawned processes and optional -network namespace are not equivalent to the production OCI controller, evidence chain, -or pristine reference. `--allow-unsafe-no-isolation` is explicitly unsafe and must never -be used for crownable work. +`scan` and `verify` are contributor diagnostics. Contributor-controlled matched A/B +profiling may load candidate code in model workers, but its process boundary is not +equivalent to the production OCI controller, evidence graph, or pristine reference. No +contributor-controlled execution path is acceptable for crownable work. ## Explicit nonclaims diff --git a/docs/validator-guide/arena-service.md b/docs/validator-guide/arena-service.md index 2f0c3a4..e9a0ed6 100644 --- a/docs/validator-guide/arena-service.md +++ b/docs/validator-guide/arena-service.md @@ -84,7 +84,10 @@ The controller supplies a durable queue snapshot. The service returns: - `admit` when screen or qualification capacity is available; - `queue` when work should wait without losing finalized priority; or -- `hold` when queue depth, queue age, cohort size, or retry policy has crossed a bound. +- `hold` when queue depth, queue age, or cohort size has crossed an admission bound. + +Retry policy is applied after `NO_DECISION` evidence through the separate +`retry_disposition()` promotion decision; it is not an input to admission. Capacity policy must be chosen from measured operational budgets. It should include expected queue age, stage cost, cohort size, retries, and crown latency. Network fetch @@ -104,6 +107,14 @@ oversized cohort or exceeded queue bound holds it, while insufficient active qualification capacity queues it. These distinctions prevent temporary saturation from becoming a miner loss while ensuring an indefinitely overloaded service fails visibly. +A hold is durable and never disappears merely because time passed. The store exposes two +explicit reservation-level dispositions: `release_hold`, with an audited reason, and +`expire`, which is admitted only after the configured minimum age. It does not expose an +automatic TTL or a typed transition that retires an entire arena. Settlement lease expiry +and a discovery claim's configured lifetime are different state machines and do not clear +an arena hold. Operators must monitor held rows and preserve their evidence; deleting or +silently aging them out can reorder authority and is not a supported recovery path. + ## Provider interface Reviewed deployment code implements two operations: @@ -113,13 +124,14 @@ class ArenaServiceProvider: provider_digest: str def run_screen(self, manifest, stage, candidate): ... - def build_qualification(self, request): ... + def build_qualification(self, request, state=None): ... ``` -`run_screen` must return the requested stage and typed evidence. `build_qualification` -must preserve the exact promoted reservation order and qualification-policy digest while -supplying a plan factory, isolated executor, post-commit entropy provider, hidden judge, -and absolute deadline. +`run_screen` must return the requested stage and typed evidence. The service always passes +the controller-supplied `state` object to `build_qualification`; providers that do not need +it must still accept the optional parameter. The returned work must preserve the exact +promoted reservation order and qualification-policy digest while supplying a plan factory, +isolated executor, post-commit entropy provider, hidden judge, and absolute deadline. The provider runs in the trusted deployment process. It is not discovered from package entry points or submission metadata. @@ -191,6 +203,13 @@ without a Python caller injecting `ArenaServiceRegistry` fails closed. Use `--intake-only` for the public standalone workflow. A production operator must supply a reviewed provider and registry through deployment code. +The deployment API also accepts `retained_only=True` on `run_pass()` and +`run_validator()`. This mode requires an existing finalized cursor, reads a fresh +finalized head, and processes the already-durable queue without rereading or advancing +reveal history. It conflicts with `intake_only` and still requires the reviewed arena +authority needed by the work. It is intentionally not exposed as a public +`chain-validate --retained-only` flag. + The repository also does not ship a registry serialization format that turns arbitrary configuration into trusted executable authority. That omission is deliberate: the provider object, secrets, executor, and hidden judge are deployment capabilities, while @@ -218,11 +237,11 @@ arena ID alone is not enough to compare measurements after a deployment change. - Registry typing does not attest that an operator chose representative prompts or sufficient capacity. - Screens do not prove a win, grant a crown, or authorize a release. +- Screen and qualification receipts do not make optional sampled in-engine audit a + universal authority; only evidence required by the registered policy contributes to + the decision. - Multiple registered arenas do not imply their measurements can be mixed; every result remains bound to one arena and stack identity. -- The retained repository proof used test/structural providers. No production provider - for a real evaluator fleet exists in the repository as of the 2026-07-13 state of - record. Next: [Authoritative qualification](qualification.md). diff --git a/docs/validator-guide/chain-loop.md b/docs/validator-guide/chain-loop.md index bc30f9c..dfa1321 100644 --- a/docs/validator-guide/chain-loop.md +++ b/docs/validator-guide/chain-loop.md @@ -95,9 +95,9 @@ optima chain-validate \ Remove `--once` to run continuously; `--interval` controls the delay between passes. -The current command has no `--wallet`, `--hotkey`, `--ledger`, `--eval-cmd`, -`--margin`, or `--dry-run-weights` options. Those flags belonged to an earlier -prototype and must not appear in service units or runbooks. +`chain-validate` accepts only its declared intake and arena schema. Chain-signing +credentials, external evaluator commands, scoring policy, and weight publication belong +to separate authorities and must not be added to the validator-loop service. Without `--intake-only`, the CLI rejects startup unless its Python caller injects an exact `ArenaServiceRegistry` and selects a registered `--arena-id`: @@ -176,7 +176,7 @@ transport and qualification retries, cohort size, epoch cutoff, and expiry. Thes code defaults, not a promise that they suit every deployment; an operator should review them alongside arena capacity. -The default `IntakePolicy` currently sets: +The default `IntakePolicy` values are: | Bound | Default | Effect | |---|---:|---| @@ -239,7 +239,7 @@ A hold is not self-healing. Diagnose the retained `reason`, repair the authority the reviewed release/requeue API appropriate to the deployment. The store's `release_hold(...)` appends an operator reason and chooses the lane from retained publication and reproduction evidence; it does not erase prior attempts. No public CLI -currently wraps reservation-hold release, so deployment tooling must expose it under its +wraps reservation-hold release, so deployment tooling must expose it under its own access controls and audit trail. ## Incident playbook diff --git a/docs/validator-guide/fidelity.md b/docs/validator-guide/fidelity.md index b7bab77..a978f1b 100644 --- a/docs/validator-guide/fidelity.md +++ b/docs/validator-guide/fidelity.md @@ -4,12 +4,12 @@ Optima uses different quality checks at different stages. Confusing them creates security error: a useful development diagnostic is not necessarily safe as a grading oracle. -## The three layers +## The quality layers | Layer | Purpose | Crown authority? | |---|---|---| | Typed slot verification | Fast ABI, output-layout, numerical, applicability, and graph preflight | No | -| `evaluate` / `bench` KL or in-engine audit | Developer feedback and integration diagnosis | No | +| Contributor-controlled model A/B or in-engine audit | Development feedback and integration diagnosis | No | | Pristine T over sealed B/C/B′ trajectories | Candidate-free, retained production quality evidence | Yes, as one required part of qualification | ## Slot verification @@ -22,20 +22,19 @@ requires graph evidence where applicable. This catches many broken kernels cheaply, but it samples a finite contract. It cannot by itself establish end-to-end model behavior or serving quality. -## Development KL and audit modes +## Development quality evidence -`optima evaluate --fidelity-mode kl` compares aligned rollout distributions. Mean KL is -not the only diagnostic: configured tail and argmax-disagreement checks help expose sparse -corruption. KL is interpretable only when stock-vs-stock variation is within the setup's -calibration; launch nondeterminism can otherwise dominate the signal. +Engine developers may compare aligned rollout distributions, task behavior, and sampled +dispatcher calls while integrating a contribution. KL-like measurements are interpretable +only when stock-vs-stock variation is within the setup's calibration; launch +nondeterminism can otherwise dominate the signal. Tail and argmax-disagreement measures +help expose sparse corruption that a mean alone can hide. -`--fidelity-mode audit` adds an untimed candidate launch that samples live dispatcher -calls and compares them with an in-engine stock calculation. This is valuable for -debugging nondeterministic stacks, but the candidate process contains the audit machinery -and cannot be the final authority against a hostile engine. - -Framework-mode diagnostics use token matching and request diagnostic no-egress isolation. -They do not make engine-wide setup a registered target and do not produce a crown. +An in-engine audit samples live candidate calls and compares them with a stock +calculation. It is valuable for debugging nondeterministic stacks, but the candidate +process contains the audit machinery and cannot grade a hostile engine. Framework-mode +token matching has the same limitation. These checks are engineering tools, not a +registered target or crown authority. ## Pristine T @@ -86,8 +85,8 @@ identity, and reproduction evidence. mathematical proof. - Calibration is specific to the registered runtime, model, hardware, and workload and must be redone when those identities change. -- Diagnostic KL/audit output is useful evidence for engineers but has no settlement - effect. +- Contributor-controlled KL, task, or audit output is useful evidence for engineers but + has no settlement effect. - Passing fidelity does not establish license, security review, or release readiness. See [Evidence and replay](../security/evidence.md) for how raw quality products are diff --git a/docs/validator-guide/first-hour.md b/docs/validator-guide/first-hour.md index ea1c4ff..b2c731b 100644 --- a/docs/validator-guide/first-hour.md +++ b/docs/validator-guide/first-hour.md @@ -206,9 +206,9 @@ result = run_validator( ) ``` -Do not substitute a shell evaluator, dynamic import path, `optima evaluate`, or a fake -provider that declares success. A commissioned provider must return real typed screen -evidence and construct the real isolated B/C/B′/T qualification work. +Do not substitute a shell command, dynamic import path, or fake provider that declares +success. A commissioned provider must return real typed screen evidence and construct the +isolated B/C/B′/T qualification work. ## 8. Observe one complete reservation lifecycle @@ -227,9 +227,9 @@ second PASS uses the reproduction lane and different authority, selection, attem report, and evidence identities. Confirm that both evidence roots reopen before a settlement candidate is admitted. -This repository's retained closure proof used structural qualification fixtures; it did -not supply a deployable production provider or a current empirical GPU crown. Your -commissioning run must not relabel those fixtures as production performance evidence. +Structural qualification fixtures can validate transitions, independence checks, and +evidence reopening. They cannot satisfy the production-provider, GPU-performance, or +calibration requirements of this commissioning step. ## 9. Commission the signer separately @@ -251,9 +251,13 @@ can be sealed, and an existing signed release can be verified against an externa pinned public key. Release construction, signing, registry publication, and serving-host launch are reviewed programmatic deployment APIs, not public CLI commands. -Do not call the validator commissioned merely because unit and slice tests pass. As of -the 2026-07-13 state of record, the actual joined OCI double-build and TP8 serve smoke had -not run, and no production `ArenaServiceProvider` shipped with the repository. +Do not call the validator commissioned merely because unit and slice tests pass. The +release plane requires the complete +[publication checklist](../engine/release-workflow.md#publication-checklist): clean-wheel +closure, provider-specific native proof, builder-authenticated reproducible OCI outputs, +closed effective serving policy, serving at the approved topology, and bound serve +receipts. The evaluation plane separately requires a reviewed production +`ArenaServiceProvider`; neither boundary is supplied by passing the library test suite. ## Production-readiness checklist diff --git a/docs/validator-guide/overview.md b/docs/validator-guide/overview.md index 8b78f05..bd55cc0 100644 --- a/docs/validator-guide/overview.md +++ b/docs/validator-guide/overview.md @@ -126,14 +126,15 @@ Read [The chain loop](chain-loop.md), [Arena service](arena-service.md), | Materialize a release build context | `optima release-context` | | Construct, sign, publish, or start a release | Reviewed programmatic APIs; no public construction CLI is bundled | -`optima verify`, `optima evaluate`, and `optima bench` are developer diagnostics. -They are useful before submission and during integration, but their output is not a -crown, a settlement record, or weight authority. See [Running diagnostics](running-evals.md). +`scan` and `verify` are contributor diagnostics. Contributor-controlled matched A/B +profiling is useful before submission and during integration, but its output is not a +crown, a settlement record, or weight authority. See +[Contributor profiling](running-evals.md). ## Durable state -Production referee state lives in `FinalizedIntakeStore`, not the legacy JSON ledger. -The store binds its database to a chain genesis hash and netuid, records finalized +Production referee state lives in `FinalizedIntakeStore`. The store binds its database to +a chain genesis hash and netuid, records finalized priority, and carries each reservation through fetch, screening, qualification, reproduction, settlement, and weight-publication state. WAL mode, full synchronous writes, a process lock, and explicit restart recovery make partially completed work @@ -163,8 +164,8 @@ Use the authority boundary to decide who absorbs a failure: | Weight readback missing or divergent | Publication `held` | Preserve journal, audit chain state, append an explicit release only after review | | Release verification or serve receipt failure | No rollout | Quarantine artifact/image; do not fall back silently to stock serving | -The JSON `commit`, `reveal`, `ledger`, and `legacy-settle --development-only` commands -remain local simulation tools. They do not describe production economics. +Developer-local state and profiler output do not describe production economics and cannot +replace any durable intake, qualification, settlement, or weight-publication product. ## What the implementation does not claim diff --git a/docs/validator-guide/qualification.md b/docs/validator-guide/qualification.md index 5cf6dfe..86ddedd 100644 --- a/docs/validator-guide/qualification.md +++ b/docs/validator-guide/qualification.md @@ -4,8 +4,8 @@ Qualification asks a narrow question: does one exact submitted delta improve one evaluation stack, in one registered arena, at acceptable quality? The production answer comes from a causal B/C/B′/T protocol executed by a trusted host -controller. It does not come from `optima evaluate`, an in-engine self-audit, a miner -report, or an arbitrary evaluator command. +controller. It does not come from a local diagnostic launch, an in-engine self-audit, a +miner report, or an arbitrary evaluator command. ## Identities before execution @@ -96,19 +96,24 @@ bookends and one pristine reference lifetime where the policy permits. This is a operational optimization, not a semantic relaxation: - every C remains one exact marginal delta; -- candidate ordering is derived from post-commit entropy; -- the selection receipt covers the exact chosen prompts/tasks and ordering; +- candidate ordering is sealed in finalized cohort/plan order before entropy is observed; +- post-commit entropy selects the hidden prompt/task work, and the selection receipt binds + that choice without reordering candidates; - drift outside calibration produces `NO_DECISION`; and - retained evidence must still support each candidate independently. The contract does not require four cold model loads per chain reveal. It requires the evidence represented by the four arms to remain causally and cryptographically separable. -For registered cohorts, an infrastructure failure before a complete attempt produces -`NO_DECISION` for every affected reservation and a persisted bisection plan. Subsequent -passes halve the cohort to isolate a poisoning or resource-sensitive candidate in -logarithmic retry groups. A per-candidate `NO_DECISION` after a complete shared attempt is -requeued individually. Neither mechanism changes finalized arrival order. +For registered cohorts, a recognized cohort-level factory, runner, raw-speed, +outer-session, or OCI-backend failure that the intake boundary normalizes into a +qualification failure product produces `NO_DECISION` for every affected reservation and a +persisted bisection plan. Subsequent passes halve the cohort to isolate a poisoning or +resource-sensitive candidate in logarithmic retry groups. A per-candidate `NO_DECISION` +after a complete shared attempt is requeued individually. Other provider, controller, or +evidence-publication exceptions abort the pass and recover through controller +restart/hold handling rather than this typed batch product. Neither mechanism changes +finalized arrival order. ## Gates and three-way decisions @@ -172,38 +177,64 @@ independence fields differ: | | Selection-secret commitment digest | | | Selection evidence digest | +“Independent” in this state-machine contract means those seven digest distinctions. The +schema does not attest that the attempts used different operators, hosts, organizations, +or infrastructure failure domains; a deployment that requires those properties must bind +and audit them separately. + After the first PASS, the same reservation goes through the five non-crown screens again in the reproduction lane. Only the second PASS creates a `SettlementCandidate`. Before settlement, the store requires exactly two retained qualification rows, reopens both -evidence roots, confirms both dispositions still carry PASS authority, and binds a new +attempt references from their recorded store roots, confirms both dispositions still +carry PASS authority, and binds a new settlement-evidence receipt. The slower passing speedup is used even if the primary was faster. ## Reopen and regrade -The final report is derived from retained raw artifacts and calibration manifests. The -system can reopen those products, verify their hashes and identities, and rederive graph, -speed, and quality decisions. A summary JSON line without its evidence is not authority. +Full regrade requires more than the persisted attempt reference. The caller must +reconstruct the exact `CausalQualificationInput`, including prepared plans, candidate +authorities, graph/calibration references and requirements, runtime policy, reference +authority, and commitment. SQLite's authority manifest and +`CohortQualificationAttempt` bind identities but do not embed that complete private +provider/plan object. Settlement restart authenticates attempt bytes and stored PASS +dispositions; it does not invoke the full causal regrader. + +The final report is derived from the serialized attempt, referenced graph/quality +artifacts, and calibration manifests. Reopen can regrade graph and raw quality evidence. +Speed regrading uses the retained `SpeedWitness`: three aggregate B/C/B′ +`ChargedExecutionRate` rows containing launch/session identities and +conditioning/timed/charged token counts and intervals. It recomputes rates and the frozen +speed decision from those aggregates; it does not reconstruct them from raw session +frames. A summary JSON line without these typed products is not authority. See [Evidence and replay](../security/evidence.md) for retention and audit requirements. -An authoritative attempt is not one file. The retained closure includes the authority -manifest, selected plan and commitment/entropy receipts, B/C/B′ lifecycle and device -receipts, graph or discovery-execution evidence, raw charged-speed batches, pristine T -session exchanges, raw quality products, per-candidate reports, and the enclosing attempt -artifact. Settlement keeps references to both complete attempt roots. +An authoritative attempt is not one headline. Durable authority includes the authority +manifest; selected plan and commitment/entropy/selection receipts; referenced graph or +discovery-execution evidence; the aggregate speed witness; the pristine-T execution +witness and raw quality artifact/binding; per-candidate reports; and the enclosing attempt +artifact. Settlement keeps references to both attempt roots. + +The live outer session validates richer B/C/B′ protocol frames, lifecycle order, device +state, and cleanup before constructing that attempt. Those raw frames and per-arm device +samples are not serialized into `CohortQualificationAttempt`. The aggregate speed witness +must not be documented as raw batch retention or as proof that a later audit can replay the +original timing frames. Reopening verifies hashes and expected bindings before grading. If the attempt artifact -reopens but a referenced raw product does not, authority is still incomplete. Operators -must retain the complete evidence graph and test restores, not merely archive the final -report digest. +reopens but a referenced graph, calibration, or raw quality product does not, authority is +still incomplete. Operators must retain every referenced evidence-store object and test +restores, not merely archive the final report digest. If policy requires raw B/C/B′ frame +replay, the attempt schema must first be extended to retain and bind those products. ## Qualification incident handling | Incident | Required disposition | |---|---| | Candidate engine exceeds deadline or violates protocol with attributable evidence | Grade under the frozen requirement; `FAIL` only when attribution is complete | -| Worker, Docker, GPU, driver, or evidence publication failure | `NO_DECISION`; repair infrastructure and use bounded retry | +| Recognized worker, Docker, GPU, driver, plan, runner, or raw-speed authority failure | `NO_DECISION`; repair infrastructure and use bounded retry | +| Evidence-store publication failure | Abort the pass; recovery holds an interrupted `qualifying` row as `controller_restart_during_qualifying` rather than manufacturing a typed `NO_DECISION` | | B/B′ drift exceeds calibration | `NO_DECISION`; do not increase the candidate's denominator or tune the bar after seeing C | | Candidate survives past the pre-T quiescence proof | Abort authority; never launch T into the contaminated lifetime | | T identity/session mismatch | `NO_DECISION`; T cannot be replaced with B′ or a candidate-side audit | @@ -225,9 +256,9 @@ qualification under the registered policy. against GPU, driver, kernel, or container-runtime compromise. - A crown records measurement and attribution. It does not satisfy integration, license, provenance, maintainability, or release review. -- The current repository does not ship the production provider that constructs this work - for a real arena. Its retained structural two-PASS closure fixture is not an empirical - GPU crown or a current production calibration. +- Deployment must supply a reviewed production provider that constructs this work for the + registered arena. Structural two-PASS fixtures can test the authority path but cannot + establish an empirical GPU crown or production calibration. Next: [Settlement and weights](settlement-and-weights.md). diff --git a/docs/validator-guide/releases.md b/docs/validator-guide/releases.md index 813ee8f..9dce639 100644 --- a/docs/validator-guide/releases.md +++ b/docs/validator-guide/releases.md @@ -28,6 +28,14 @@ flowchart LR Every arrow is an explicit authority transition. None is automatic, and none may be reconstructed from a human-readable release note alone. +!!! danger "Use the canonical acceptance gate" + The programmatic APIs expose release-construction and host-verification primitives; + they do not imply that the pinned implementation is deployable. Production acceptance + requires the complete [publication checklist](../engine/release-workflow.md#publication-checklist), + including clean-wheel closure, provider-specific native proof, effective serving-policy + closure, builder-output binding, and descriptor/session-bound receipts. Consult the + dated [State of record](../reference/state-of-record.md) before operating this sequence. + ## Integration gate Before a crown enters `EngineReleaseManifest`, review must establish at least: @@ -94,7 +102,7 @@ bytes. The command prints the content digest, receipt digest, and receipt path. - `ServeSpec` with digest-pinned base image, OCI platform, model mount, TP size, and restricted engine arguments/environment. -The current runtime distribution is `optima-engine` version `0.0.1`. Its deterministic +The runtime distribution is `optima-engine` version `0.0.1`. Its deterministic artifacts are: ```text @@ -159,7 +167,8 @@ enters it. ## Registry and serving host -The reviewed programmatic host path can: +After the canonical acceptance gates are supplied, the reviewed programmatic host +primitives participate in a workflow that must: 1. build and push the context twice with no cache and no build network; 2. require identical registry manifest digests; @@ -168,8 +177,14 @@ The reviewed programmatic host path can: 5. pull and create a digest-pinned serving container with read-only root, tmpfs, seccomp, no-new-privileges, dropped capabilities, read-only model mount, and reviewed labels; 6. verify policy again before start; and -7. require active, routed, and completed receipts for every released slot and rank, with - no load-failure or fallback receipts. +7. require descriptor/session-bound active, routed, and completed receipts for every + released slot and rank, with no load-failure or fallback receipts, plus per-rank AOT + load/invocation coverage for sealed direct artifacts. + +The generic functions do not themselves provide all of these end-to-end authorities. +In particular, a matching registry readback must be authenticated against each builder's +emitted output, and the generic serve-receipt verifier does not bind receipt frames to a +release/session or enforce AOT receipt coverage. Serving uses host network and host IPC intentionally because this is a reviewed product container, not an untrusted candidate evaluator. The release runtime verifies the signed @@ -178,31 +193,36 @@ before executing SGLang in a fresh interpreter. ## Release operating sequence -Use a two-person or otherwise independently reviewed release procedure: +Use this sequence only after the canonical checklist and state ledger show that every gate +exists. Require a two-person or otherwise independently reviewed release procedure: 1. **Freeze inputs.** Pin the active crown, integration record set, engine tree, model receipt, native publication, upstream revision, SGLang version, base-image digest, platform, seccomp bytes, reference/calibration manifests, and `ServeSpec`. 2. **Prepare twice-derived evidence.** Run programmatic release preparation and verify - source/wheel reproducibility, SBOM/provenance closure, integrated-only manifest, native - tree binding, and model receipt. + source/wheel reproducibility, clean-wheel import/entrypoint closure, + SBOM/provenance closure, integrated-only manifest, provider-specific native + profile/index/ABI proof, native tree binding, and model receipt. 3. **Sign offline or in an isolated release authority.** Publish the canonical descriptor and signature. Distribute the expected public key and descriptor digest through a separate trusted channel; never learn trust from `release.sig.json` itself. 4. **Verify on the build host.** Run `release-verify`, then `release-context` into a new, empty destination. Do not reuse a mutable prior context. 5. **Build and publish twice.** The reviewed host API performs two no-cache builds with no - build network, pushes both, and requires the same registry manifest digest before + build network and requires one registry manifest digest. Separately authenticate that + each admitted registry object came from its corresponding builder output before producing signed reproducibility authority. 6. **Authorize the exact container.** Bind registry repository/digest, local image inspection, release descriptor, external key, platform, labels, model bind, seccomp, - device request, command, and environment. + device request, effective command/environment, loader/network policy, and authenticated + management routes. 7. **Create, verify, then start.** The container is created with the reviewed restrictions; host policy is checked again before start. Startup must fail closed when release-required seam activation cannot be proven. -8. **Gate rollout on receipts.** Require active, routed, and completed receipts for every - released slot and expected rank, with no load failure or fallback receipt. Retain the - exact image, authorization, launch, and receipt products. +8. **Gate rollout on receipts.** Require release/session-bound active, routed, and completed + receipts for every released slot and expected rank, with no load failure or fallback + receipt. For sealed direct artifacts, also require bound per-rank AOT load and invocation + coverage. Retain the exact image, authorization, launch, and receipt products. Use canary/percentage rollout and ordinary service health controls in addition to these integrity checks. Optima's release receipts do not replace latency, capacity, availability, @@ -225,7 +245,7 @@ or loading the last crowned proposal directly. | Release key suspected compromised | Stop new rollouts under the key | Rotate external trust anchor and re-sign/re-authorize through reviewed procedure | | Runtime health regression after valid startup | Route back to a prior verified release | Preserve SLO evidence; integrity success does not block operational rollback | -Keep release keys off validator, weight-signer, evaluator, and serving hosts. The current +Keep release keys off validator, weight-signer, evaluator, and serving hosts. The double-build API signs its container-reproducibility attestation, so the release/publish workflow must provide signing authority at that step. Isolate and audit that narrowly scoped key use rather than leaving the private key resident on a general-purpose build @@ -233,7 +253,7 @@ worker. ## Public-surface limitations -The repository currently provides public CLI commands to provision a model, verify an +The public CLI provides commands to provision a model, verify an existing signed release, and create its build context. It does **not** provide a public CLI to: @@ -256,13 +276,15 @@ not imply that `release-context` publishes or deploys anything. SLOs or universal workload performance. - Release verification is chain-independent and does not re-adjudicate the crown. -!!! warning "Current validation boundary (2026-07-13)" - The release implementation is unit- and slice-tested, but the retained closure did - **not** run an actual joined no-cache double OCI build followed by TP8 serving and - receipt verification. The retained crown used for release-structure proof was a - synthetic 100-ppm fixture, not an empirical RTX/B300 win. No production arena provider - ships in the repository. Treat the host and serving sequence above as required future - operational validation, not a completed performance claim. +!!! warning "Production validation gate" + Unit and slice tests do not authorize a release. Production validation requires clean + wheel installation, provider-specific native reopening, two builder-authenticated + no-cache OCI outputs with matching registry identity, closed effective serving policy, + serving at the approved topology, and bound receipt verification for the exact signed + release/session. Structural crown fixtures can exercise release plumbing but cannot + supply empirical qualification or production calibration. Consult + [State of record](../reference/state-of-record.md) before claiming that any operational + gate has been completed. ## Source anchors diff --git a/docs/validator-guide/running-evals.md b/docs/validator-guide/running-evals.md index b2ce261..ea8b1be 100644 --- a/docs/validator-guide/running-evals.md +++ b/docs/validator-guide/running-evals.md @@ -1,135 +1,158 @@ -# Running diagnostics +# Verification and diagnostics -Optima exposes three developer-facing checks. They answer different questions and none -of them is authoritative settlement evidence. +Optima exposes two local contribution checks: `scan` and `verify`. Complete-engine +throughput and quality decisions belong to a registered validator arena; the CLI does +not provide a local qualification substitute. -| Command | Question | Typical environment | +| Check | Question answered | Authority | |---|---|---| -| `verify` | Does each declared variant satisfy its typed slot ABI and reference checks? | CPU or GPU; spawned worker processes | -| `evaluate` | On this development setup, how do baseline and candidate throughput and diagnostic fidelity compare? | GPU model launch | -| `bench` | Does the candidate regress selected task accuracy, KL diagnostics, or noisy throughput? | GPU model launch | +| `scan` | Does the declared bundle tree satisfy the static intake policy? | Local admission diagnostic | +| `verify` | Do applicable variants satisfy the registered component ABI, reference, and graph checks? | Local component diagnostic | +| Arena screen | Can the exact materialized delta pass static, build, ABI, graph, and abbreviated-serving gates? | Promotion eligibility only | +| B/C/B′/T qualification | Does the exact marginal delta clear execution, speed, and pristine-quality policy? | One qualification decision | +| Independent reproduction | Do two separately bound PASS attempts reopen and agree? | Settlement prerequisite | -The production path instead uses a registered arena service, complete materialized engine -trees, no-egress OCI sessions, pristine T, retained evidence, and independent -reproduction. See [Qualification](qualification.md). +Unknown bundles still execute code during verification. Run them only inside the minimum +[hostile-code isolation boundary](../security/isolation.md#operator-requirements) used for +the relevant device and contribution class; a Python environment or ordinary container is +not an adequate boundary for untrusted native GPU code. -!!! warning - These commands are for code development and integration. Do not run unknown bundles - on a machine that lacks an appropriate outer sandbox. `--allow-unsafe-no-isolation` - explicitly weakens a diagnostic and must never be part of a crownable workflow. - -## Op-level verification - -CPU smoke test: +## Static policy scan ```bash -python -m optima.cli verify ./my_bundle --device cpu --dtype float32 +python -m optima.cli scan ./my_bundle ``` -GPU verification: +The command parses the manifest, applies the Python policy to every declared and vendored +`.py` file, recognizes only manifest-declared CUDA sources and dependency patches, and +rejects symlinks, binary artifacts, undeclared executable material, and files outside the +benign metadata allowlist. It scans an extracted bundle tree, not a transport archive. +Archive extraction and resource limits belong to finalized intake. A clean result is +defense in depth, not a sandbox or a correctness proof. -```bash -python -m optima.cli verify ./my_bundle \ - --device cuda --dtype bfloat16 --seed 17 -``` +## Component verification -Collective slots need the intended rank count: +CPU smoke: ```bash -python -m optima.cli verify ./my_collective_bundle \ - --device cuda --world-size 4 --dtype bfloat16 +python -m optima.cli verify ./my_bundle --device cpu --dtype float32 ``` -Use `--model ` when a validator model profile changes activation or low-bit -correctness semantics. A context-inapplicable variant is not treated as a successful -verification. - -Verification checks all declared variant domains for ambiguity before loading a row. It -also exercises required graph capture/replay for graph-bound block and collective slots. -CPU fallback is correctness-only and cannot establish GPU graph coverage. - -## End-to-end throughput diagnostic - -Always point at model bytes controlled by the operator: +CUDA verification: ```bash -python -m optima.cli evaluate ./my_bundle \ - --model /models/example-model \ - --num-prompts 32 \ - --max-new-tokens 64 +python -m optima.cli verify ./my_bundle \ + --device cuda --dtype bfloat16 --seed 17 \ + --model ``` -The command runs a development baseline/candidate comparison and prints a diagnostic -score that is explicitly not recorded. Keep CUDA graphs enabled for realistic graph-path -testing; `--disable-cuda-graph` is a debugging control, not a scoring configuration. - -Useful controls include: - -- `--tp-size`, backend selections, and engine JSON for the intended development setup; -- `--timed-iters` and `--warmup-iters` for thermal/noise diagnosis; -- `--max-running-requests` for a serving-like concurrency bound; -- `--prompt-seed` for repeatable local investigation; and -- `--fidelity-mode kl|audit` for the legacy diagnostic quality path described below. - -### Long-prefill diagnostics - -`--input-len` generates deterministic long inputs. The timed iterations replay the same -prompts, so the radix cache can turn later iterations into decode-only measurements. Pair -long-prefill testing with a reviewed engine configuration that disables the radix cache: +Distributed verification: ```bash -python -m optima.cli evaluate ./my_prefill_bundle \ - --model /models/example-model \ - --input-len 8192 \ - --engine-kwargs-json '{"disable_radix_cache": true}' +CUDA_VISIBLE_DEVICES=0,1,2,3 \ +python -m optima.cli verify ./my_collective_bundle \ + --device cuda --world-size 4 ``` -That flag combination remains a development diagnostic; production workload regimes are -bound by the arena manifest. - -## Capability diagnostic - -```bash -python -m optima.cli bench ./my_bundle \ - --model /models/example-model \ - --benchmarks gsm8k,mmlu \ - --samples 128 +Use `--model` when a validator model profile changes activation or low-bit semantics. +Use `--tp-size` for capability-aware non-collective routing and `--world-size` for the +actual distributed verifier. Verification rejects ambiguous variant domains before +loading candidate code and treats a context-inapplicable row as not exercised, not as a +pass. + +CPU verification covers component math only. CUDA verification additionally exercises +registered graph capture/replay behavior. Neither establishes live SGLang seam coverage, +end-to-end throughput, pristine reference quality, or production isolation. + +## Performance development + +The repository has no public complete-engine benchmark command and no contributor command +that materializes a validator's incumbent evaluation stack. Local performance work is +therefore a contributor-controlled experiment built with external launch and profiling +tooling. It becomes comparable to a named arena only when the operator has published the +complete contract and the contributor can reproduce every disclosed input. + +Before launching, freeze these inputs: + +| Input class | Required identity or value | +|---|---| +| Candidate | Canonical bundle content hash, target ID, selected variant, and the exact source/native publication under test | +| Comparison stack | Exact incumbent manifest and engine-tree identities; if they are unavailable, identify the substituted stock/reviewed baseline and do not call the result an arena reproduction | +| Runtime | Container/base digest, SGLang revision, model content identity, launch arguments, environment, dtype, graph mode, and cache policy | +| Hardware | GPU model, driver/runtime, visible device set, TP/EP/DP degrees, rank mapping, clocks/power policy, and interconnect topology | +| Workload | Prompt/request corpus identity, batching/concurrency policy, warmup, measured iterations, token/work accounting, and timing boundary | +| Activation | Evidence that C activates only the selected target delta and that B/B′ use the identical stack without it | + +Use the deployment's ordinary SGLang launcher and profiler only after those identities are +fixed. Materialize a fresh process lifetime for each arm; do not toggle candidate state in +one long-lived engine. The bracket is: + +```text +B = exact incumbent before the candidate +C = the same stack with only the selected target delta replaced +B′ = the exact incumbent after the candidate +local_speedup = candidate_rate / mean(baseline_before_rate, baseline_after_rate) ``` -Small sample counts are smoke tests, not a meaningful accuracy floor. `bench` reports -baseline/candidate task results, KL diagnostics, and noisy throughput. It does not emit a -qualification report or evidence set suitable for settlement. - -## Diagnostic fidelity modes - -`evaluate` retains two legacy development modes: - -- `kl` aligns baseline and candidate rollout distributions and checks configured mean, - tail, and argmax-disagreement thresholds. It is informative only when repeated stock - launches are sufficiently deterministic for that calibration. -- `audit` runs an extra untimed candidate launch and samples dispatcher calls against an - in-engine stock calculation; rollout KL becomes advisory. Because this executes in the - candidate engine, it is not the production grading authority. - -Framework-mode diagnostics switch to token matching and imply the diagnostic no-egress -namespace request. They do not authorize engine-wide setup as a registered economic -target. Cross-cutting proposals belong to the separate discovery lane. - -Production fidelity comes from candidate-free pristine T after C is destroyed. See -[Fidelity and quality authority](fidelity.md). - -## Interpreting results +For each arm, retain startup/activation evidence, complete warmup, then collect the same +number of measured samples under the same charged-work definition. Keep CUDA graphs in the +declared state and reject the bracket when B/B′ drift is comparable to the claimed gain. +For a prefill target, disable cache behavior that would silently convert repeated inputs +into cache-hit or decode work. A profiler range may explain a mechanism, but throughput +must use the declared end-to-end charged boundary rather than a hand-selected kernel span. + +The local result record should contain: + +- every frozen identity from the table above; +- raw per-sample B, C, and B′ work counts, durations, and rates; +- warmup count, measurement order, failure/retry history, and B/B′ drift; +- the speedup formula, charged-work denominator, and any exclusion rule; +- activation/fallback evidence for the selected slot on every expected rank; and +- tool versions plus an immutable location or digest for the raw logs. + +If any required identity, raw sample, or activation signal is missing, label the result a +profiling observation rather than a matched bracket. Do not substitute the validator's +private workload, calibration, or incumbent identities with guesses. + +This bracket is engineering evidence only. A contributor-controlled run cannot provide +finalized intake identity, validator-owned materialization, hidden work, frozen +calibration, no-egress worker authority, or a validator-bound durable attempt with its +aggregate speed witness and referenced graph/quality/T products. The production attempt +also validates richer raw B/C/B′ frames and device state in-run, although those frames are +not serialized into `CohortQualificationAttempt`. A local run cannot provide the required +second PASS or its seven digest-distinctness products. + +## Reading validator outcomes + +- `PASS` means one complete attempt cleared every registered gate. +- `FAIL` requires complete evidence of a candidate-attributable violation. +- `NO_DECISION` covers infrastructure, drift, missing authority, or incomplete evidence + and is eligible only for bounded retry policy. +- `reproduction_pending` means the first PASS is retained; it is not a crown. +- settlement requires two matching contribution identities with distinct authority, + plan, attempt, report, and selection evidence. + +Never infer rejection from absence in `chain-status`; that command sees public chain +state, not the validator's private lifecycle database. + +## Evidence scope + +| Evidence | Establishes | Does not establish | +|---|---|---| +| Clean scan | Static policy accepted the declared tree | Safety or correctness | +| CPU verify | Exercised component reference checks | CUDA graphs or performance | +| CUDA verify | Exercised component and registered graph checks | Model integration or crown authority | +| Local B/C/B′ bracket | A development performance hypothesis | Registered arena identity or quality authority | +| One arena PASS | Complete qualification under one authority | Settlement | +| Two reopened independent PASSes | Settlement eligibility for that exact context | Integration or release readiness | -- A quality failure is useful development feedback, not a settlement event. -- A noisy B/C/B′ diagnostic may print `NO-DECISION`; gather better evidence rather than - converting it to a win or loss. -- A local speedup does not establish the registered arena, target identity, graph proof, - pristine quality evidence, whole-stack regression, or independent reproduction. -- A diagnostic pass never changes the SQLite evaluation stack or reward claims. +See [Qualification](qualification.md), [Fidelity](fidelity.md), and +[Evidence and replay](../security/evidence.md). ## Source anchors -- [CLI implementations](https://github.com/latent-to/optima/blob/main/optima/cli.py) +- [CLI](https://github.com/latent-to/optima/blob/main/optima/cli.py) +- [Static scanner](https://github.com/latent-to/optima/blob/main/optima/sandbox.py) - [Typed verifier](https://github.com/latent-to/optima/blob/main/optima/verify.py) -- [Development throughput/KL diagnostic](https://github.com/latent-to/optima/blob/main/optima/eval/throughput_kl.py) -- [Development capability diagnostic](https://github.com/latent-to/optima/blob/main/optima/eval/capability.py) +- [Distributed verifier](https://github.com/latent-to/optima/blob/main/optima/verify_collective.py) +- [Qualification runner](https://github.com/latent-to/optima/blob/main/optima/eval/qualification_runner.py) diff --git a/docs/validator-guide/scoring.md b/docs/validator-guide/scoring.md index 7952801..52edf7a 100644 --- a/docs/validator-guide/scoring.md +++ b/docs/validator-guide/scoring.md @@ -18,10 +18,12 @@ For a registered target, the validator constructs: - C: that same stack with one registered target replaced; and - B′: a second independent launch of the exact incumbent. -The charged execution rate is recomputed from bounded raw batch evidence. The controller -fixes prompt batches and token budgets, validates the token numerator, and includes -registered conditioning and timed intervals. Candidate-reported aggregate throughput is -not accepted as authority. +During execution, the controller fixes prompt batches and token budgets, validates bounded +batch frames and token numerators, and constructs charged rates that include registered +conditioning and timed intervals. The durable `SpeedWitness` retains three aggregate +`ChargedExecutionRate` rows; reopen recomputes tokens/second and the frozen decision from +their counts and intervals, not from raw session frames. Candidate-reported aggregate +throughput is never accepted as authority. The speed estimate is conceptually: @@ -62,11 +64,18 @@ settled_speedup = min(primary_speedup, reproduction_speedup) There is no single-pass fast path to a crown. -## Target-level settlement +Here “independent” means the seven required authority, plan, attempt, report, commitment, +secret-commitment, and selection-evidence digests differ. It does not by itself prove +separate operators, hosts, or infrastructure failure domains. -Settlement evaluates candidates against one exact incumbent stack. Stale candidates are -held. Among current registered candidates, deterministic ordering selects the highest -conservative speedup, using finalized arrival order as the tie-break. +## Settlement cohort over one incumbent authority + +The store leases one economically unblocked group sharing a qualification authority and +one exact incumbent stack. Stale candidates are held. Across all current registered rows +in that leased group—even rows for non-overlapping targets—the planner selects one winner +by conservative speedup and uses finalized arrival order as the tie-break. The shared +incumbent advances once, so every other current row is held for a fresh qualification +against the new stack rather than treated as an independent per-target argmax. The winning transaction may emit crown, retirement, neutralization, adoption, and stack transition events. Atomic targets explicitly displace overlapping singleton targets; @@ -85,6 +94,11 @@ improvement = speedup_ppm - 1_000_000 credit = floor(improvement * half_life / (half_life + age_blocks)) ``` +`speedup_ppm` is the floor of the conservative Decimal speedup multiplied by 1,000,000. +Standing-claim age begins at the proposal's finalized submission block, which settlement +stores as `crowned_block`; discovery lifetime likewise begins at that submission block via +`awarded_block`. Qualification or settlement delay never resets reward age. + An active atomic target suppresses overlapping singleton families. Packaging, integration, and release records do not create additional families. Discovery bounties are non-renewable, expire, and share a policy-bounded pool. @@ -98,15 +112,16 @@ details. ## What a result means -A crown means: under the registered arena, workload, calibration, and two independent -attempts, the exact delta improved the exact incumbent with acceptable measured quality. +A crown means: under the registered arena, workload, calibration, and two attempts meeting +the seven digest-distinctness checks, the exact delta improved the exact incumbent with +acceptable measured quality. It does not mean: - the contribution improves every model, topology, or traffic mix; - the measured speedup is a service-level capacity guarantee; - the proposal is licensed, maintainable, reviewed, or ready to ship; or -- a diagnostic `score` printed by `evaluate` or `bench` has economic effect. +- any score produced outside registered, retained qualification has economic effect. ## Source anchors diff --git a/docs/validator-guide/settlement-and-weights.md b/docs/validator-guide/settlement-and-weights.md index 2878848..9f811c1 100644 --- a/docs/validator-guide/settlement-and-weights.md +++ b/docs/validator-guide/settlement-and-weights.md @@ -52,11 +52,19 @@ remain blockers; a later fast result cannot jump them. The default lease is 30 b Expiry returns it to pending with a higher lease generation, preventing a worker holding the old lease from committing. -Immediately before commit, the controller refreshes the finalized clock. Inside the -transaction the store verifies that the lease has not expired, the incumbent stack and -event-journal head have not advanced, economic blockers have not changed, both retained -evidence products are still byte-identical, and the plan exactly equals a freshly -recomputed plan. Any disagreement aborts the transaction. +The controller reads a fresh finalized height immediately before requesting each lease +and refuses a regressed clock. It refreshes the height again immediately before commit. +Inside the transaction the store verifies that the lease has not expired, the incumbent +stack and event-journal head have not advanced, economic blockers have not changed, both +retained evidence products are still byte-identical, and the plan exactly equals a +freshly recomputed plan. Any disagreement aborts the transaction. A pass with no pending +settlement work does not make these extra finalized-height reads. + +Lease expiry is not arena retirement. Intake reservations have explicit `release_hold` +and minimum-age `expire` transitions, but no automatic TTL and no typed transition that +retires an entire arena. Held state can continue to block later overlapping work until an +operator completes an evidence-preserving reservation disposition; deleting rows is not +a recovery mechanism. ## Deterministic plan @@ -91,8 +99,10 @@ The event journal is append-only and digest chained. Event types have distinct j catalog, not manifest order. - `ADOPTION` and `STACK_TRANSITION` record the exact new evaluation manifest/tree and advance its generation together. -- `HOLD` records a stale-incumbent candidate or a deterministic conflict loser without - mutating the stack. +- `HOLD` records stale rows and every current registered non-winner without mutating the + stack: stale-incumbent rows use `stale_incumbent`, overlapping losers use + `conflict_lost`, and non-overlapping current losers use `incumbent_advanced` because the + winning transition advanced the shared incumbent. - `DISCOVERY_BOUNTY` creates only the bounded discovery claim; it has no stack transition. Do not infer event meaning from a miner bundle name or the final row status. Reopen the @@ -104,8 +114,8 @@ Every active registered target defines one family. A singleton target owns its s atomic target owns its complete member set and suppresses explicitly overlapping singleton families while active. -Credit comes from independently reproduced marginal improvement, expressed in integer -parts per million: +Credit comes from marginal improvement accepted by a two-PASS pair satisfying the seven +digest-distinctness checks, expressed in integer parts per million: ```text improvement = speedup_ppm - 1_000_000 @@ -117,6 +127,12 @@ This reciprocal decay reaches half the original credit after one half-life. Reti neutralized contributions receive no standing credit. Engine-stack packaging, integration, and release do not create additional reward families. +`speedup_ppm` is `floor(conservative_decimal_speedup × 1_000_000)`. Settlement initializes +the standing claim's `crowned_block` from the proposal's finalized submission block, not +from settlement time, so delayed qualification does not reset reward age. Discovery +claims use the same finalized submission block as `awarded_block` for their bounded +lifetime. + ## Discovery bounties Discovery does not install an evaluation stack entry. A qualifying discovery may create @@ -141,6 +157,15 @@ If an active family is stale, incompatible, missing, unreopenable, or assigned t hotkey absent from the metagraph, the complete projection is held. Its share is never silently redistributed. +The bound chain observations are not one block-atomic substrate snapshot. `WeightProjection` +retains the projection-time effective block, metagraph digest, recipient vector, and state +identities. The reconciler performs validator-weight and later confirmation reads through +separate SDK calls while the chain may move. `WeightPublicationRecord` retains only the +projection digest, status chain, submission/retry/reveal/confirmation blocks, +`confirmed_last_update`, and reason; it does not serialize the pre/post readback vectors +or a post-read metagraph digest/hash. The projection digest authenticates the desired +vector and projection inputs, not the later observations used to compare or confirm it. + Projection is global across every retained arena, not “one `set-weights` call per target.” The builder requires a catalog for every evaluation stack, reopens the evidence behind every active standing and discovery claim, binds the emissions-policy digest on @@ -182,11 +207,13 @@ Real publication is fail-closed and journaled: | `held` | Authority changed, deadline expired, readback diverged, or post-submit state is unavailable | | `released` | Operator appended an audited release of a retained hold | -An SDK return value—even `submitted=True`—is not confirmation. The reconciler refreshes -the metagraph immediately, maps recipients to current UIDs, reads current validator -weights, persists intent before signing, and confirms only when the chain has -the exact recipient set, each normalized value is within the fixed -`2e-5` relative/absolute verifier tolerance, and `last_update` is new enough. +The chain helper checks the SDK response's `success` field (or the older tuple form), so +a call may return without raising and still report `submitted=False`. Even +`submitted=True` is not confirmation. The reconciler refreshes the metagraph immediately, +maps recipients to current UIDs, reads current validator weights, persists intent before +signing, and confirms only when the chain has the exact recipient set, each normalized +value is within the fixed `2e-5` relative/absolute verifier tolerance, and `last_update` +is new enough. The normal state transitions are: @@ -206,8 +233,15 @@ stateDiagram-v2 `pending` is a valid unresolved result, not success. Before its retry block, another run only observes it. At or after the deadline, absent matching readback becomes `held` rather -than blindly resubmitting. If an unresolved record belongs to a different newly computed -projection, that conflict also becomes held. +than blindly resubmitting. + +On every non-dry public invocation, `set-weights` first constructs the current projection, +then reopens the exact retained projection named by an `intent` or `pending` journal head. +It resumes that immutable vector across later chain heads when chain scope, netuid, and +signer authority still match; a restart cannot replace an in-flight vector merely because +a newly computed head would produce different weights. An authority mismatch fails +closed. A direct caller that bypasses this resume step and presents a different projection +to the low-level reconciler receives a retained hold rather than a silent replacement. The reconciler can record a preexisting chain match as `confirmed` without submitting. Conversely, it refuses a real submission when `crown_count` is zero, when the wallet @@ -248,10 +282,14 @@ authority before deciding whether a new intent is valid. | Either PASS evidence root cannot reopen | No settlement and no reward projection | Restore exact content-addressed bytes or retain hold | | Transaction fails mid-plan | SQLite rollback; no partial events/claims/stack | Diagnose, then rerun against unchanged authority | | Active claim hotkey leaves metagraph | Entire projection refused | Resolve validator-set policy; never redistribute implicitly | +| Initial projection is stale, post-read is unavailable, block/`last_update` chronology is impossible, or recipient/vector readback differs | Projection is rejected or publication is held | Refresh authority and reconcile again; do not infer a general membership-churn comparison that the reconciler does not perform | +| SDK returns an unsuccessful response without raising | `submitted=False`; authoritative readback still governs the journal | Inspect the response and chain state; do not convert lack of an exception into success | | SDK says submitted, readback absent | `pending`, then `held` at deadline | Inspect chain/extrinsic; preserve journal and append reviewed release if appropriate | | Post-submit chain authority unavailable | Immediate `held` | Restore authoritative reads before release/retry | | Previously confirmed vector changes | `held` | Treat as an incident; compare chain history and signer activity | | Emissions parameters differ from bound policy | Projection refused | Use the consensus-approved bound policy or migrate authority explicitly | +| Held reservation has no disposition | It remains durable and may keep blocking later work | Preserve and monitor it; use audited `release_hold` or minimum-age `expire`, never silent deletion | +| Arena must be retired as an authority domain | No generic transition is available | Define and implement a reviewed typed arena-retirement policy before changing economic authority | The journal and settlement tables are evidence. Back them up with SQLite-aware tooling, monitor WAL/disk health, and test restoration with evidence roots present. Never repair an diff --git a/docs/validator-guide/testnet.md b/docs/validator-guide/testnet.md index 9bcef4e..88d4ab9 100644 --- a/docs/validator-guide/testnet.md +++ b/docs/validator-guide/testnet.md @@ -27,10 +27,11 @@ The payload is limited to 1024 UTF-8 bytes and posted through the chain's native commit-reveal interface. The validator uses finalized reveal event order as priority. Proposal bytes, evaluation evidence, stack state, and release artifacts remain off chain. -The reveal event's finalized `(block, block hash, event index, event subindex, hotkey, -content hash)` becomes the reservation identity and priority. `chain-status` can show -revealed commitments before the validator has accepted them as finalized history; wait -for finality rather than using display order or local arrival time as authority. +Priority is the finalized `(block, event index, event subindex, hotkey, content hash)` +arrival key. The reservation identity is a separate canonical digest that also binds the +block hash, URL, and finalized-payload digest. `chain-status` can show revealed +commitments before the validator has accepted them as finalized history; wait for +finality rather than using display order or local arrival time as authority. ## Prerequisites @@ -163,9 +164,9 @@ validation requires reviewed Python deployment code to inject a registry/provide select the arena. See [Arena service](arena-service.md) and [Qualification](qualification.md). -Do not resurrect the removed `--eval-cmd`, `--ledger`, `--margin`, -`--dry-run-weights`, wallet, or hotkey flags on `chain-validate`. They do not exist in the -current production path. +`chain-validate` accepts only its documented intake and arena controls. External evaluator +commands, scoring policy, chain-signing credentials, and weight publication belong to +separate authorities and must not be added to the intake service definition. With an injected deployment registry, the same testnet loop may screen, qualify, retain a two-PASS pair, and settle. That does not move B/C/B′/T on chain: the chain supplies arrival @@ -192,15 +193,19 @@ optima set-weights \ --dry-run ``` -Dry run refreshes live authority and prints/reconciles the projection without creating a -publication intent. An empty intake-only test database cannot be turned into weights by -supplying arbitrary CLI values. +Dry run refreshes live authority and computes the projection without creating a +publication intent. Stable CLI output reports the projection digest, dry-run status, +chain-match result, and `submitted=False`; it does not emit the projected UID/weight +vector or every projection input. An empty intake-only test database cannot be turned +into weights by supplying arbitrary CLI values. The dry run still opens the exclusive SQLite authority and reads a live metagraph. Stop or coordinate the validator loop so it does not own the database at that moment. Preserve -the projected digest, effective block/hash, membership digest, policy digest, stack -generations, and output vector. `status=dry_run` and `submitted=false` are the expected -non-emitting result. +the printed projection digest and status. If an operational audit also requires the +effective block and hash, membership and policy digests, stack generations, or output +vector, capture the typed projection and its inputs through reviewed programmatic +instrumentation; the stock CLI does not serialize those fields to stdout. +`status=dry_run` and `submitted=False` are the expected non-emitting result. A real weight test is a materially different operation: it requires a genuine crown, persists journal intent before signing, and may submit a hotkey extrinsic. Do not remove @@ -208,18 +213,21 @@ persists journal intent before signing, and may submit a hotkey extrinsic. Do no confirmation; the exact recipient set, fixed-tolerance normalized values, and a sufficiently new chain `last_update` are required. -## What the retained testnet proof established +## Evidence scope -The 2026-07-13 closure record retained a real finalized testnet reveal through strict -HTTPS fetch, committed-tree re-hash, registered-target resolution, immutable publication, -and restart with zero new work. It also retained a live metagraph-backed weight dry run -with no extrinsic. +Executing this runbook authorizes only the products that are successfully retained and +reopened. Do not infer a stronger boundary from testnet connectivity: -The later generation-1 crown/reproduction portion was explicitly structural and used a -synthetic 100-ppm fixture. It was not a current empirical GPU win. The actual joined OCI -double-build/TP8 serving proof was not run, and the repository still ships no production -`ArenaServiceProvider`. These are commissioning obligations, not details hidden behind a -CLI flag. +| Retained product | What it can establish | What it cannot establish | +|---|---|---| +| Finalized reveal, committed-tree re-hash, immutable publication, and restart reconciliation | Chain intake, proposal identity, and durable cursor behavior | GPU qualification, settlement, or release readiness | +| Metagraph-backed weight dry run | Projection construction against live chain state without an extrinsic | Signing, submission, inclusion, or confirmation | +| Structural two-pass fixture | State-machine transitions, independence checks, evidence reopening, and settlement plumbing | Empirical GPU speedup, production calibration, or arena-provider readiness | +| Builder-authenticated reproducible OCI pair plus release/session-bound serving receipts | Release-image identity and execution through the approved serving seams for that exact session; include AOT coverage for sealed direct artifacts | Qualification authority unless the separate B/C/B′/T evidence also exists; clean-wheel, native-provider, and effective-policy gates remain separate prerequisites | + +The [state of record](../reference/state-of-record.md) identifies which of these evidence +products have been completed. Deployment must supply and commission its production arena +provider independently of the command-line intake path. ## Testnet exit criteria diff --git a/mkdocs.yml b/mkdocs.yml index 47dc4b2..15eb4b4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -105,6 +105,7 @@ nav: - Proposal to release: architecture/pipeline.md - Target and slot contract: architecture/slot-contract.md - Evaluation and release stacks: architecture/stacks.md + - Sealed direct artifacts: architecture/direct-artifacts.md - Runtime injection seam: architecture/seam.md - Release architecture: architecture/releases.md - Miner guide: @@ -131,7 +132,7 @@ nav: - Fidelity and pristine T: validator-guide/fidelity.md - Settlement and weights: validator-guide/settlement-and-weights.md - Release operations: validator-guide/releases.md - - Direct eval diagnostics: validator-guide/running-evals.md + - Verification and diagnostics: validator-guide/running-evals.md - Testnet: validator-guide/testnet.md - Optima Engine: - Overview: engine/overview.md