DEP (light): Path-independent worker admission and relay identity

[cmdy]

Area

frontend

Summary

Exclude source_path from ModelDeploymentCard::mdcsum() while retaining every other existing checksum input and admission check. Compatible workers should be able to serve one logical model from different local directories without introducing new MDC fields.

For relay identity, evaluate two alternatives: the path-independent mdcsum() (R1) and the primary display_name (R2). Both retain KV block size, query hash format, routing scope, explicit identity overrides, and isolation of rejected members. One alternative will be selected for implementation.

The admission change targets the first-wins logic introduced by PR #14632 and is associated with PR #14610.

Motivation

A model can be served by workers in multiple resource pools with different installation directories. The workers may advertise identical configuration, tokenizer, template, and auxiliary-file checksums while differing in source_path, network addresses, and capacity.

Currently, source_path participates in both mdcsum() and the relay's default cache-semantic identity. A path-only difference can therefore prevent admission or produce conflicting indexer domains. A directory is not a reliable model identity: identical weights can occupy different paths, and different machines can store different weights at the same path.

Deployment example

Region A provides the shared request entry point. Regions B and C run independent workers and connect over the network; they do not join Region A's Kubernetes cluster. All workers serve the same logical model under the same Dynamo namespace and serving endpoint, using shared NATS infrastructure.

flowchart TB
    Client["Clients"] --> Frontend
    subgraph A["Region A: Kubernetes / DGD"]
        Frontend["Dynamo frontend / router"]
        WA["H20 worker<br/>/models/example"]
        Frontend --> WA
    end
    Frontend -->|Network| WB["Region B: independent GPU machines<br/>H200 worker · /mnt/models/example"]
    Frontend -->|Network| WC["Region C: cloud deployment platform<br/>H20 worker · /srv/models/example"]

Loading

Arrows show inference traffic. In our deployment, remote workers use a custom dynamo-external-hook to establish Kubernetes-based discovery in Region A. This hook illustrates our current integration only; it is neither a prerequisite for this DEP nor a proposed Dynamo feature. The compatibility change applies to otherwise compatible workers with different model paths, regardless of how they are discovered.

sequenceDiagram
    participant Hook as dynamo-external-hook
    participant Worker as Remote worker (B or C)
    participant K8s as Region A Kubernetes API
    participant Frontend as Region A frontend
    Hook->>K8s: Create shadow Pod and headless Service
    Hook->>Worker: Inject Pod context and KUBECONFIG
    Worker->>K8s: Register DynamoWorkerMetadata
    Frontend->>K8s: Watch worker metadata
    K8s-->>Frontend: Worker discovery updates
    Frontend->>Worker: Send inference requests over the network

Loading

The hook obtains kubeconfig from a remote API and stores it in a temporary file with mode 0600. Shadow Pods use a pause container and readiness checks against remote health endpoints; the model processes remain in Regions B and C. The hook manages the shadow resources' lifecycle. This deployment-specific integration establishes discovery and leaves MDC admission and relay compatibility checks in force. Upstreaming the hook is outside this proposal.

Proposal

1. Scope and compatibility

Remove source_path from mdcsum(); retain existing grouping rules, all other checksum inputs, and first-wins admission. Keep the real path for loading, metadata resolution, and diagnostics. No new MDC fields or required artifacts are introduced.

Operators assign distinct model/deployment identities to distinct weight variants. Metadata checksums establish the covered configuration compatibility, not weight equality. Removing path equality can admit identical-metadata workers previously separated by directory; paths never reliably established weight identity.

2. Admission fields

Retain current encoding, ordering, and normalization for these inputs:

Retained input	Reason / example
display_name, aliases	Preserve model registration names and existing alias-order semantics.
model_info checksum	Different config.json content can describe different architectures.
tokenizer checksum	Different vocabularies can encode the same text differently.
prompt_formatter, chat_template_file checksums	Different templates can produce different prompts.
gen_config checksum	Different defaults can change behavior when request parameters are omitted.
Sorted extra_files basename/checksum pairs	Processor configuration and required filenames affect loading. Preserve the existing set of advertised extras.
prompt_context	Additional context changes rendered input.
Effective context length	A request accepted at 128K must not reach a 32K worker. Retain runtime → architecture → legacy → zero resolution.
kv_cache_block_size	Different sizes change cache-block boundaries.
worker_type, needs	Preserve serving-role and readiness dependencies.
router_config	Preserve a consistent WorkerSet routing policy.
Explicit indexer_identity	Preserve declared semantic and scope isolation.
Tower/connector LoRA flag and Qwen/Nemotron video contracts	Preserve existing multimodal cache and prompt-expansion distinctions.

Excluded input	Reason / example
source_path — newly excluded	Identical models can reside under /models/... and /mnt/....
Metadata URLs and sizes	Existing artifact checks compare checksums rather than hosting location.
Worker IDs, addresses, bootstrap endpoints	Replicas require distinct identities and connections.
KV capacity, batching limits, DP placement, topology	Heterogeneous workers retain individual scheduling constraints.
Extra-file enumeration order	Sorting removes directory traversal order.

All exclusions except source_path preserve current behavior. Per-worker routing, capability, and transport validation remains in force.

3. Relay identity

Derivation

Replace the default card.source_path() material with one selected alternative:

R1 = ["dynamo/mdcsum/no-source-path/v1", card.mdcsum()]
R2 = ["dynamo/model-name/v1", card.name()]

R1 uses the checksum from Section 2. R2 uses the validated primary display_name.

Use existing length-framed encoding, validated block size, and query hash-format version to derive the cache-semantics ID with BLAKE3 truncated to 16 bytes. Preserve routing-scope derivation and domain composition.

flowchart TB
    R1["R1: marker + mdcsum"] --> Selected["Selected default material"]
    R2["R2: marker + display_name"] --> Selected
    Selected --> Semantics["Cache semantics ID<br/>Material + block size + hash format"]
    Semantics --> Domain["Indexer domain"]
    Scope["Routing scope ID<br/>Namespace + component + endpoint"] --> Domain

Loading

Only one alternative is used. enable_eagle-derived query format remains an independent input. Explicit semantics replaces default material under the existing algorithm; explicit routing_scope independently overrides its dimension. Worker ownership, LoRA bindings, and transport checks remain unchanged. Domain equality does not authorize KV transfer.

Alternatives and selection criteria

	R1: checksum-derived	R2: name-derived
Primary rationale	Reuse existing metadata distinctions as an additional relay isolation boundary; avoid a second metadata projection.	Separate admission compatibility from logical cache identity; avoid coupling to unrelated configuration and checksum evolution.
Same name, different metadata	Produces different domains for covered differences.	Produces the same domain when scope and KV semantics match; membership must exclude incompatible workers.
Unnecessary separation	Roles, aliases, routing settings, context limits, generation defaults, and LICENSE can split domains.	These changes do not independently change the domain. Primary-name changes do.
Version coupling	New checksum inputs or normalization differences can change domains across readers. Explicit scope changes can also affect semantic identity through the checksum.	Domain derivation is independent of checksum evolution; admission/cohort agreement is still required.
Replacement lifecycle	Some configuration changes create new domains; generation isolation remains required.	Same-name replacements can reuse domains; stale generations must be retired explicitly.
Selection requirement	Validate checksum agreement and conservative splits against disaggregated and explicit-identity workloads. Exclude frontend-local overrides from identity derivation.	Verify that existing admission and lifecycle guarantees exclude rejected workers and stale state after same-name replacement.

R1 favors conservative metadata partitioning; R2 favors stable logical identity. Neither verifies weights or replaces membership validation. Select one algorithm before implementing PR 2.

4. Compatibility

Preserve existing model loading, metadata resolution, and multimodal preprocessing behavior. Validate mixed-version relay compatibility and pool lifecycle behavior when changing the default identity algorithm.

5. Implementation plan and PR dependencies

PR	Scope	Completion criterion
PR 1 — admission (#14610)	Remove source_path from mdcsum(); preserve existing guards; address direct path-consumer regressions.	Supported path-only differences pass admission, incompatible arrivals remain rejected, and existing serving paths do not regress. Relay-dependent cross-path support is not yet claimed.
PR 2 — relay identity	Implement the selected R1/R2 algorithm with regression, compatibility, and end-to-end validation.	Cross-path workers contribute valid KV state and participate in routing; mixed-version compatibility and pool lifecycle checks pass.

flowchart LR
    P1["PR 1: admission"] --> P2["PR 2: relay identity<br/>End-to-end support"]

Loading

PR 2 builds on PR 1. PR 1 can merge independently if it preserves existing relay behavior; otherwise, the changes must land together.

6. Validation

Area	Required evidence
Admission	Different real paths with identical covered metadata join; config/tokenizer/extra-file checksum, context-limit, and block-size mismatches remain rejected without evicting healthy members.
Path consumers	Metadata resolution and affected multimodal preprocessing remain correct.
Identity	Fixed hash vectors, path-only equality, block/hash-format and scope separation, unchanged explicit overrides.
Admission / relay integration	Verify that workers rejected by admission do not affect relay membership or the active cache-state view. Reuse existing admission and lifecycle guarantees.
R1	Participating readers compute identical checksums; additional role/configuration splits preserve supported deployments.
R2	Same-domain incompatible arrivals are excluded; same-name replacement rejects delayed old-generation events.
End-to-end	Two different model directories provide usable KV state and receive requests through one frontend; worker cache ownership remains correct.
Compatibility	Mixed-version relay operation and pool lifecycle behavior remain correct when the identity algorithm changes.

References

• PR #14610: exclude source path from compatibility identity
• PR #14632: first-wins MDC admission
• Community discussion
• Model card and checksum
• Admission controller
• Relay domain resolution
• Relay discovery
• Identity encoding