upgrade: support vLLM 0.28 on NVIDIA and Ascend 910C

[physics31415926]

Summary

Extend PR #484's vLLM 0.28 NVIDIA upgrade to Huawei Ascend 910C while preserving the NVIDIA path. This branch is rebased directly onto PR #484's final commit 8a946e5, targets 0.4.0-dev, and contains 58 linear commits above 0.4.0-dev@f053d2f (35 inherited from #484 and 23 Ascend commits), with no merge commits.

NVIDIA uses the regular vLLM 0.28 package. Non-NVIDIA installations use an empty-device vLLM build; the Ascend image uses vllm==0.28.0+empty and pairs it with torch-npu, FlagTree/Triton, FlagGems, and cann-shmem. The existing shared CI workflow is retained and extended through the platform configuration; this PR does not add a separate Ascend workflow.

The Ascend implementation is validated with Qwen3.6, DeepSeek-V4-Flash ModelSlim W8A8, and the approximately 720.9 GiB GLM-5.2 ModelSlim W8A8 checkpoint. The final GLM run loads all 182 shards at TP16 and completes both short and multi-token prompts through the native Ascend dynamic W8A8 grouped-MoE path.

Merge ordering: keep this PR open until NVIDIA PR #484 lands. Then rebase this branch onto the updated 0.4.0-dev and rerun the relevant checks before considering merge.

Changes

Files / area	Change and purpose
Branch history	Rebase the Ascend work directly onto PR #484 final commit 8a946e5 and use 0.4.0-dev as the target branch. The final range is linear and preserves all #484 NVIDIA changes.
README.md, pyproject.toml	Set the compatibility baseline to vLLM 0.28.0. Document regular vLLM for NVIDIA and the empty-device build for non-NVIDIA hardware.
docker/ascend/**, docker/build.sh	Build vLLM 0.28.0+empty on the Ascend A3 base and package the validated torch-npu, FlagTree/Triton, FlagGems, cann-shmem, and runtime constraints. Document the published harbor.baai.ac.cn/plugin image.
vllm_fl/platform.py, vllm_fl/worker/**, vllm_fl/compilation/**	Adapt platform, worker, V1/V2 runner, compilation, graph, cache, and packed-GDN contracts to vLLM 0.28. NVIDIA keeps V2 by default; Ascend uses the validated eager path.
vllm_fl/dispatch/backends/vendor/ascend/**, vllm_fl/dispatch/config/ascend.yaml	Port Ascend attention, causal convolution, GDN/FLA, multimodal attention, rotary embedding, vocabulary-parallel embedding, cache/block-size handling, and FlagTree compatibility to vLLM 0.28.
vllm_fl/ops/fused_moe/**, vllm_fl/dispatch/backends/vendor/ascend/impl/grouped_moe.py	Migrate to vLLM 0.28 modular MoE and run unquantized Ascend experts through native grouped matmul. Flatten expert_map lookup before index_select so FlagGems correctly handles 2-D routing tensors and negative/non-local routes.
vllm_fl/quantization/w8a8/moe_experts.py	Replace the failing FlagGems fused-MoE Triton path on Ascend with native CANN dynamic quantization, grouped INT8 matmul, SwiGLU, requantization, route weighting, and reduction. Keep NVIDIA on vLLM/FlagGems and retain the reference fallback for unsupported bias layouts.
vllm_fl/models/deepseek_v4.py, vllm_fl/quantization/modelslim_w8a8.py	Add the Ascend DeepseekV4ForCausalLM adapter and ModelSlim dynamic-INT8 linear/routed-expert support.
vllm_fl/models/glm_moe_dsa.py, vllm_fl/patches/ascend_glm_dsa.py, vllm_fl/__init__.py	Register native GlmMoeDsaForCausalLM on Ascend, add the eager short-context dense-attention fallback, restore the original Transformers config after construction, and ignore only the top-level MTP-only rot.weight tensor when loading the target model.
GLM ModelSlim W8A8 support	Resolve GLM packed parameter names, static asymmetric W8A8 attention linears, dynamic routed/shared experts, TP slicing, offsets, dequant scales, and row-parallel bias handling with native torch-npu kernels.
.github/configs/**, .github/scripts/**, .github/workflows/**	Keep the shared workflow, enable Ascend and 0.4.0-dev, select the published image, install pytest reporters, isolate device-specific collection, and cap the 910C offline profile at the serving-validated 4096-token shape.
Security fixes	Resolve all six GitHub Advanced Security review threads: normalize imports, replace empty exception handlers with suppress, and remove the Sunrise-to-worker reverse import while preserving PTPU backend normalization.
Tests and examples	Add focused Ascend operator, routing, quantization, GLM loader/model, multimodal, runner-selection, and CI-matrix coverage; add an Ascend multimodal smoke example.

MetaX, GCU, and other non-NVIDIA vendor edits in this diff are compatibility-only changes and were not runtime-tested by this PR.

Environment

Component	Version / configuration
Target and lineage	0.4.0-dev@f053d2f; exact PR #484 ancestor 8a946e5; final head 1a30017; 58 linear commits, 0 merge commits
Ascend image	harbor.baai.ac.cn/plugin/vllm-plugin-fl:ascend-vllm0.28.0-a3-ci-20260917; manifest sha256:e6af662ee688fdd62a2404e25d7b9c1a5b8b197f75839c32e37a7dca1a19dbdb
Base image	quay.io/ascend/vllm-ascend:v0.20.2rc1-a3
Ascend runtime	vLLM 0.28.0+empty; PyTorch 2.10.0+cpu; torch-npu 2.10.0; Python 3.11.15
Ascend Triton	FlagTree 0.6.2a1+ascend3.5 provides Triton 3.5.1; standalone triton and triton-ascend packages are rejected
Ascend libraries	FlagGems 5.0.2 at 3b406c36212744b98b9720bf6d0a5387c09fe96b; cann-shmem 1.6.0; NumPy 1.26.4
Installation layout	The published image embeds the final plugin wheel and runtime in site-packages. Shared CI intentionally overlays the PR checkout with pip install --no-build-isolation --no-deps -e .; independent image validation uses no plugin source mount or PYTHONPATH, and pip list --editable is empty.
Final image build source	Git archive of 1a30017; SHA-256 45c0ccecccadd869f4e9402d14596e33d2424f385a5f89bd2df9f19bb485e49d
Final embedded wheel	vllm_plugin_fl-0.0.0+g1a30017-py3-none-any.whl; SHA-256 9dd4979c345af8e98fe6b92b5abcbffa3c4214675741ac9b9819d805fab12ebc
Runtime-code equivalence	1a30017 differs from TP16-tested 888540f only in .github/configs/ascend.yml and docker/ascend/README.md; no Python or runtime code changed.
Ascend hardware	Huawei Ascend 910C (910C_174); TP1, TP2, TP8, and TP16 validation
NVIDIA baseline inherited from #484	vLLM 0.28.0; PyTorch 2.13.0+cu129; FlagTree 0.6.2a1 / Triton 3.6.0; FlagGems 5.3.5+g34c6d2ce4; A800 and H100 x8

Test results

Scope	Test / configuration	Result
History	merge-base, exact #484 ancestry, commit-range and merge audit	PASS — 58 linear commits (35 #484 + 23 Ascend), 0 merges
Static checks	Ruff check/format, compileall, YAML parsing, shell syntax, and git diff --check	PASS
Final GitHub Advanced Security	Code Scan run 35192448121 on 1a30017; CodeQL and Dependency Review; review-thread audit	PASS — both jobs green; all six review threads resolved; no new finding
Final shared CI	Run 35192448776 on 1a30017, using the published 20260917 image	IN PROGRESS — lint, build, matrix discovery, Ascend setup, and CUDA setup passed; Ascend and CUDA unit jobs are queued for self-hosted runners; no failure
Runtime-code shared CI	Run 35184739395 on 888540f	ASCEND PASS — setup, unit, functional, offline inference E2E, serving E2E, and benchmark all passed; CUDA unit remained queued for a self-hosted runner
Final packaged image	Exact base digest plus commit-labelled wheel; raw manifest and fresh digest pull; OCI revision; non-editable site-packages import; no plugin source mount/PYTHONPATH; empty editable list; normal vLLM import entry points	PASS — manifest digest matches; OCI revision 1a30017f5b82446fc1e2f18ff0e1f8d6831a0001; vLLM 0.28.0+empty; plugin 0.0.0+g1a30017
Final focused regression	W8A8 MoE, ModelSlim W8A8, GLM model/fallback/loading, and runner selection from the final image's installed wheel	PASS — 56 passed, 4 warnings in 4.36s
Ascend W8A8 route smoke	FlagGems enabled; NPU/int32 expert_map[6]; NPU/int32 routing [32,2] with negative, local, and non-local routes; native dynamic quant + two grouped matmuls	PASS — 36 local routes, counts [16,20], output [32,128] BF16
Native grouped W8A8 numerical smoke	M=6/E=2 and M=32/E=4, including zero-token experts, dynamic quant, BF16 scale, SwiGLU and two grouped matmuls	PASS — M=32 final max/mean absolute error 4.4778/0.2061 versus reference max 1527.95 (about 0.29% max relative)
GLM-5.2 checkpoint metadata	Audit 176,587 tensor entries: FLOAT 1,420; W8A8 1,665; W8A8_DYNAMIC 173,502	PASS — zero unknown quantization types; attention/routed/MTP assignments match the implementation
GLM-5.2 TP loader and kernels	Real parameter shapes at TP16 ranks 0 and 15; static asymmetric W8A8 quant/matmul on 910C; rank-zero/nonzero row-parallel bias	PASS — TP slices and dtype conversions match; kernel max CPU-reference error about 4.9e-4
GLM-5.2 W8A8 final Python workload	Exact g888540f wheel; approximately 720.9 GiB checkpoint; BF16; ModelSlim W8A8; TP16; eager; max length 128; two prompts (2 and 27 input tokens)	PYTHON WORKLOAD PASS — harness emitted GLM52_TP16_PASS and completed normal model shutdown after 182/182 shards; weights 745.02s; engine init 1029.027s; generation about 15s. Native grouped W8A8 MoE avoided the prior fatal compiler path; a separate FlagGems dynamic-quant compile warning fell back as designed. The enclosing shell wrapper then returned 127 on an unrelated CRLF tail after inference had completed; the normalized wrapper passes bash -n.
GLM-5.2 final outputs	Short prompt generated one special/termination token; 27-token compatibility prompt generated 8 tokens	PASS — second decoded output: Careful compatibility testing across hardware back
GLM-5.2 capacity	KV cache and sampled HBM during final TP16 run	PASS — 25,088 KV tokens; 196x concurrency for 128-token requests; stable sampled HBM 50,173–51,185 MiB/card; all 16 devices released after shutdown
DeepSeek-V4-Flash ModelSlim W8A8	TP8, BF16 KV cache, max length 128, one active sequence; all 70 shards	PASS — generated Paris. The capital from The capital of France is
Qwen3.6-35B-A3B offline	Exact CI image; TP2 eager; 4096-token profile batch	PASS — 1 passed in 626.96s; generated output beginning with Paris
Earlier Ascend regression (pre-rebase; implementation replayed unchanged)	561-test unit suite; functional operators; graph/cache; distributed/HCCL; Qwen3-4B, Qwen3.6-27B text/image, Qwen3.6-35B-A3B text/image, ERNIE 4.5 0.3B	PASS — 561 passed, 9 skipped; functional 7/7; graph/cache 6/6; distributed 10/10; TP2 and TP8 HCCL passed
NVIDIA validation inherited from #484	465-test suite; Qwen3-8B, DeepSeek-V2-Lite-Chat, Qwen3-0.6B, Qwen3.8-27B, Qwen3.6-35B-A3B, Qwen3.5-0.8B, MiMo-7B-RL, Gemma3n-E2B-it; DeepSeek-V4-Flash FP8 on H100 x8	PASS — 465 passed, 23 skipped; model inference results recorded in #484

All model inference runs use VLLM_PLUGINS=fl and USE_FLAGGEMS=1.

Failures reproduced and fixed

Reproduced failure	Root cause	Fix and verification
GLM checkpoint stopped after 181/182 shards on rot.weight	The checkpoint contains a top-level MTP rotation tensor that the target non-MTP model does not own	Skip only exact top-level rot.weight; nested/similar names remain loadable. Final run loaded 182/182.
GLM prefill failed at M=32 in FlagGems fused_moe_kernel with DiscreteMaskAccessConversion / MLIRCompilationError	The Ascend Triton compiler cannot lower that fused W8A8 MoE shape	Route Ascend dynamic W8A8 experts through native CANN dynamic quant, grouped matmul and SwiGLU. Final TP16 generation passed.
FlagGems intercepted expert_map[2-D topk_ids] and failed with a 1-D/2-D gather mismatch	Advanced indexing was lowered to a gather contract with mismatched dimensions	Flatten the index, use torch.index_select, restore the original shape, and preserve negative routes. Unit and device route smoke passed; final GLM separately passed the native MoE path.
Qwen3.6-35B-A3B offline profile failed with 507015 at the vLLM 0.28 default 8192-token profile shape	The offline profile exceeded the 910C shape validated by the serving path	Cap the offline CI profile at 4096 tokens; the exact post-fix case passed in 626.96s.

Known limitations

• Ascend serving is validated in eager mode. Compiled/static-graph serving is outside this PR.
• GLM-5.2 uses the dense-attention fallback on Ascend because sparse MLA is unavailable. It requires --max-model-len <= index_topk (2048 for the tested checkpoint).
• Static ModelSlim W8A8 routed experts are rejected. The tested GLM-5.2 checkpoint uses dynamic W8A8 for all routed/shared experts and static W8A8 only for supported attention linears.
• DeepSeek-V4-Flash was validated at TP8 with BF16 KV cache, a 128-token maximum model length, and one active sequence. Long-context and MTP speculative decoding remain future work.
• The available ERNIE 4.5 21B-A3B checkpoint was incomplete (8 of 9 shards), so it was not tested.

Checklist

• Existing validated suites pass
• Tests cover the vLLM 0.28 and Ascend changes
• Documentation and packaged-image guidance are updated
• Existing shared CI is used for Ascend validation