Skip to content

feat(glm5): add GLM5.3-Flash H100 FlagOS support - #454

Open
CherryLemon wants to merge 36 commits into
flagos-ai:mainfrom
CherryLemon:day0-glm53
Open

CherryLemon wants to merge 36 commits into
flagos-ai:mainfrom
CherryLemon:day0-glm53

Conversation

@CherryLemon

Copy link
Copy Markdown
Collaborator

PR Category

Core

PR Type

New Features | Improvements | Performance

Description

Add GLM5.3-Flash H100 FlagOS support for the vLLM 0.24.0 ABI. This PR is the model-specific layer on top of the shared day0-common branch; the branch is intentionally runnable as-is while the common stack is reviewed separately.

Changes

  • Add GLM5.3 configuration, model, multimodal processor, and vLLM 0.24 compatibility patches.
  • Add portable sparse MLA/indexer and KPool kernels, MLA prefill, safe KDA, and MHC broadcast paths.
  • Make empty-build execution allocator-safe for CUDA graph capture and provide fallbacks when native DeepGEMM or stock indexer paths are unavailable.
  • Route empty-build vision attention and MoE alignment through FlagGems.
  • Add graph-boundary, KPool, MHC, processor, provider, and portable-path unit coverage.

Testing

  • H100 TP16 service path was exercised with effective FULL_AND_PIECEWISE graph mode; mixed batches used PIECEWISE and decode batches used FULL graphs.
  • Strict 1k-input/1k-output serving request completed successfully on the empty-build path.
  • Added tests under tests/unit_tests/glm5_next and recorded the graph/portable compatibility coverage.

Stacking

The model delta is reviewable with day0-common...day0-glm53. Please land/rebase it after the shared day0-common changes are accepted.

Checklist

  • I have run the relevant serving checks
  • I have added tests for my changes (if applicable)
  • I have updated the documentation (if applicable)

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants