Skip to content

[CPU:Bugfix] Preserve C4 quantization alignment across RVV chunks - #5024

Draft
luke-opus wants to merge 1 commit into
alibaba:masterfrom
luke-opus:feature/rvv-quant-vl
Draft

luke-opus wants to merge 1 commit into
alibaba:masterfrom
luke-opus:feature/rvv-quant-vl

Conversation

@luke-opus

@luke-opus luke-opus commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The RVV quantization and dequantization loops reuse a C4-periodic scale/zero-point template starting at channel zero on every iteration. RVV permits a non-maximal VL when VLMAX < AVL < 2*VLMAX, so an iteration can consume a non-multiple of four elements and make the next iteration use the wrong channel parameters.

For example, with VLEN=128 and the legal VL=ceil(AVL/2) choice, quantization of 12 elements in e32m2 consumes six elements first. The next iteration starts at channel two but reuses channel zero's scale. Dequantization has the same problem for 20 elements in e32m4.

Cap the requested AVL at the template's VLMAX in both loops. Full chunks then have a C4-aligned length, and the final chunk is exact. Scalar-parameter quantization and the packing/dispatch contracts are unchanged.

The RVV specification allows this VL choice. QEMU's rvv_vl_half_avl=true makes the failure reproducible without relying on a particular processor's VL policy.

Validation

  • Full static MNN and run_test.out cross-build/link with Clang 22.1.8, C++11, rv64gcv/lp64d, RVV on and transformer-fuse off. Changed lines checked with clang-format 17 and git diff --check.
  • Executed the production kernels in QEMU 11.1.2 at VLEN 128/256/512/1024 with default and half-AVL policies, including all-ones agnostic tails/masks. All 1,040 channel-alignment cases pass in each configuration (all four parameter modes, 0–129 C4 quads, output guards). With half-AVL enabled, the baseline fails respectively 192/189, 189/183, 183/171 and 171/147 quantization/dequantization cases. Inputs are integral to isolate channel alignment from the independent rounding bug.
  • RISC-V Linux/QEMU, VLEN 128/512 and half-AVL: verified actual RVV function-table selection; direct cases and the existing op/ConvInt8/depthwise test pass at 1 and 4 threads. Five single-batch, seven-channel synthetic quant/dequant graphs also pass against a scalar-kernel control at both thread counts.

Minimal example at VLEN=128 and rvv_vl_half_avl=true: quantize 12 values all equal to 1, with scale={1,2,3,4}, zero point 0 and quanParamVec=1. Element 6 should be 3 but the baseline uses 1. Dequantizing 20 int8 values all equal to 1 with the same parameters similarly gives 1 instead of 3 at element 10. Both are corrected by capping AVL.

Limitations: a separate two-batch/seven-channel synthetic graph fails identically with RVV and with both quant/dequant pointers replaced by the existing scalar kernels; its root cause remains under investigation outside this diff. This PR also intentionally leaves the independent Float2Int8 rounding bug to the companion PR #5023. Combining both fixes passes all 29,120 direct cases across the eight VLEN/VL-policy configurations. This is simulator evidence, not physical-board or pretrained-model validation; the PR remains a draft and makes no performance claim.

Module

CPU / RISC-V RVV

Type

  • Feature
  • Bugfix
  • Perf
  • Refact
  • Style
  • Doc
  • Test
  • Chore

Checklist

  • Commit message follows [Module:Type] Description format
  • Code compiles without errors
  • Tested on relevant platform(s) — QEMU execution completed; physical RISC-V board and pretrained-model validation pending
  • No unrelated format or style changes included

Shared iOS CI failure

The iOS job fails on a pre-existing Xcode project reference to the absent express/module/MoEModule.cpp. The RVV changes do not modify that project or file. The independent build fix is #5026, which passes the same iOS framework build script locally on Xcode 26.6 / iPhoneOS SDK 26.5. The failed runs on this PR have not been marked as passing; they need a new build with the project fix included.

@luke-opus luke-opus added type:bug 功能缺陷 module:quantization 量化相关 labels Oct 22, 2026
@wangzhaode wangzhaode self-assigned this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:quantization 量化相关 type:bug 功能缺陷

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants