fix(frontend): accept null optional completion parameters

[cmdy]

Overview:

Some OpenAI-compatible clients serialize unset optional request parameters as explicit null. This is accepted by SGLang-compatible request schemas, but Dynamo currently validates several frontend passthrough fields as concrete values before a request reaches the backend.

For example, the following request fragment currently returns HTTP 400 with Validation: cache_salt must be a string:

{
  "cache_salt": null,
  "stop_token_ids": null,
  "stream_options": {
    "include_usage": null,
    "continuous_usage_stats": null
  }
}

This PR treats these explicit nulls as unset values for both /v1/chat/completions and /v1/completions, allowing the request to proceed with Dynamo's existing defaults.

Details:

• Treat null as omitted for the six recognized fields in PASSTHROUGH_EXTRA_FIELDS.
• Accept missing or null boolean members inside a stream_options object and default them to false.
• Preserve current validation for unknown fields and malformed non-null values.
• Preserve the existing non-streaming and force-include-usage policies.

Passthrough fields

The flattened extra-field deserializer now removes a field only when both conditions are true:

1. Its value is null.
2. Its name is present in PASSTHROUGH_EXTRA_FIELDS.

The affected fields are:

• cache_salt
• stop_token_ids
• detokenize
• allowed_token_ids
• bad_words_token_ids
• logprob_token_ids

This normalization happens before validation and backend extraction, so a null value has the same meaning as an omitted optional field. Valid non-null values continue to be forwarded without modification. Invalid non-null values continue to return a validation error.

Unknown fields are deliberately retained, even when their value is null. They continue to be rejected by default or ignored when DYN_IGNORE_OPENAI_FE_UNSUPPORTED_FIELDS is enabled.

Stream options

The chat and text completion handlers deserialize requests through a typed wrapper that captures stream_options separately from the flattened completion request. This allows omitted or null values for:

• stream_options.include_usage
• stream_options.continuous_usage_stats

Both fields default to false. Explicit true and false values are preserved. Invalid types remain rejected. The request body is deserialized once without materializing an intermediate JSON tree, and duplicate-field validation remains intact.

Compatibility behavior

Request input	Behavior after this PR
Recognized passthrough field set to null	Treated as omitted
Recognized passthrough field with a valid non-null value	Validated and forwarded unchanged
Recognized passthrough field with an invalid non-null value	Rejected with HTTP 400
Unknown field set to null	Follows DYN_IGNORE_OPENAI_FE_UNSUPPORTED_FIELDS
stream_options omitted or set to null	Existing optional behavior is preserved
Boolean member omitted or set to null inside stream_options	Defaults to false
Boolean member explicitly set to true or false	Preserved
Boolean member set to an invalid type	Rejected with HTTP 400

The normalization is implemented at the Dynamo OpenAI frontend boundary, so all configured backends receive the same normalized request behavior.

Where should the reviewer start?

1. lib/llm/src/protocols/openai/validate.rs: deserialize_extra_fields contains the allowlisted null normalization and its protocol regression tests.
2. lib/llm/src/http/service/openai.rs: parse_completion_json_request contains the stream_options fallback normalization and endpoint-level parsing tests.
3. lib/llm/src/protocols/openai/chat_completions.rs and lib/llm/src/protocols/openai/completions.rs: both request types apply the custom flattened-field deserializer.

Validation

• cargo test -p dynamo-llm --no-default-features --locked --lib protocols::openai:: — 471 passed, 0 failed.
• cargo test -p dynamo-llm --no-default-features --locked --lib http::service::openai::tests::test_parse_ — 12 passed, 0 failed.
• cargo fmt --all -- --check
• git diff --check

The regression coverage verifies both chat and text completions, all six passthrough fields, unknown-null preservation, valid and invalid non-null values, null and missing stream flags, explicit stream flags, usage-policy overrides, and malformed stream option types.

Related Issues

🚫 This PR is NOT linked to an issue:

• Confirmed — no related issue

Summary by CodeRabbit

• Bug Fixes
  • Improved compatibility when processing OpenAI completion and chat-completion requests with missing or null usage-streaming options.
  • Preserved clear errors for malformed option values and invalid request structures.
  • Improved handling of invalid UTF-8 and unescaped control characters in supported payloads.
  • Unsupported fields are now validated consistently, with appropriate errors for malformed or unknown values.
  • Null-valued passthrough fields are handled more reliably without affecting valid non-null fields.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi cmdy! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a088e3ba-988f-4f22-95f7-68efdbd7260a

📥 Commits

Reviewing files that changed from the base of the PR and between b6a5b70 and 38dbb16.

📒 Files selected for processing (4)

• lib/llm/src/http/service/openai.rs
• lib/llm/src/protocols/openai/chat_completions.rs
• lib/llm/src/protocols/openai/completions.rs
• lib/llm/src/protocols/openai/validate.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

Changes

OpenAI request parsing

Layer / File(s)	Summary
Extra-field validation lib/llm/src/protocols/openai/chat_completions.rs, lib/llm/src/protocols/openai/completions.rs, lib/llm/src/protocols/openai/validate.rs	Request deserialization validates unsupported fields. Recognized null passthrough fields are omitted, while unknown fields remain subject to validation.
Tolerant stream-option parsing lib/llm/src/http/service/openai.rs	Completion and chat-completion handlers use tolerant parsing for omitted or null usage flags. Invalid option shapes and schema errors still return errors. Tests cover normalized flags and existing tolerant parsing cases.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 38dbb

The request parsing changes preserve validation behavior while adding the intended null normalization; no merge-blocking risk is currently identified.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 4 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: the frontend now accepts null optional completion parameters.
Description check	✅ Passed	The description includes all required template sections and provides clear details, reviewer guidance, related-issue status, and validation results.

---

_{Comment @coderabbitai help to get the list of available commands.}

[furionw]

/ok to test 5f6478a

[furionw]

Thank you, @cmdy . This seems reasonable to both SGLang and vLLM.

[furionw]

/ok to test 757d223

[luckyq]

Null handling looks correct, and existing validation is preserved. I haven’t run the tests.

[rmccorm4]

[P2] Avoid copying escaped request text through another flatten layer

This still buffers the request twice: NvCreate*Request already contains flattened fields, so the additional wrapper traverses Serde's outer Content through ContentRefDeserializer, copying owned escaped strings into another buffer. The wrapper runs even when stream_options is absent. At this head, a 15 MiB decoded prompt containing newlines (18 MiB JSON body) raises peak live parsing allocation from 46 MiB to 61 MiB compared with direct deserialization of the same request type. That is an extra 15 MiB per concurrently parsing request; the plain-text control was effectively unchanged. These are allocator measurements in a debug build, excluding the input body, not RSS or throughput estimates.

Normalize the two booleans in ChatCompletionStreamOptions with #[serde(default, deserialize_with = "super::deserialize_null_as_default")], reuse the existing null-default helper in the protocol crate, and remove this wrapper so both handlers use parse_json_request directly. I validated that change locally with a patched protocol dependency: 750 tests passed, and the null-containing stream-options probe returned identical parsed output with peak parsing allocation back at 46 MiB. Shipping it needs the corresponding protocol release and dependency bump.

Follow-up to the earlier allocation finding: the explicit JSON-value retry is gone, but the extra payload copy remains in this replacement.

[cmdy]

Thanks for the detailed measurements. Agreed—the wrapper should be removed. The protocol-side null handling is already available in dynamo-protocols 6.0.1 through frontend-crate #239, but this PR is still on 5.4.1 and #14755 owns the protocols 6 migration. I’ll wait for #14755 to land, then update this branch, remove the wrapper, restore direct parse_json_request usage in both handlers, and rerun the targeted tests.