Repository navigation
Conversation
4 of 12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The RVV quantization and dequantization loops reuse a C4-periodic scale/zero-point template starting at channel zero on every iteration. RVV permits a non-maximal VL when
VLMAX < AVL < 2*VLMAX, so an iteration can consume a non-multiple of four elements and make the next iteration use the wrong channel parameters.For example, with VLEN=128 and the legal
VL=ceil(AVL/2)choice, quantization of 12 elements in e32m2 consumes six elements first. The next iteration starts at channel two but reuses channel zero's scale. Dequantization has the same problem for 20 elements in e32m4.Cap the requested AVL at the template's VLMAX in both loops. Full chunks then have a C4-aligned length, and the final chunk is exact. Scalar-parameter quantization and the packing/dispatch contracts are unchanged.
The RVV specification allows this VL choice. QEMU's
rvv_vl_half_avl=truemakes the failure reproducible without relying on a particular processor's VL policy.Validation
run_test.outcross-build/link with Clang 22.1.8, C++11,rv64gcv/lp64d, RVV on and transformer-fuse off. Changed lines checked with clang-format 17 andgit diff --check.op/ConvInt8/depthwisetest pass at 1 and 4 threads. Five single-batch, seven-channel synthetic quant/dequant graphs also pass against a scalar-kernel control at both thread counts.Minimal example at VLEN=128 and
rvv_vl_half_avl=true: quantize 12 values all equal to 1, withscale={1,2,3,4}, zero point 0 andquanParamVec=1. Element 6 should be 3 but the baseline uses 1. Dequantizing 20 int8 values all equal to 1 with the same parameters similarly gives 1 instead of 3 at element 10. Both are corrected by capping AVL.Limitations: a separate two-batch/seven-channel synthetic graph fails identically with RVV and with both quant/dequant pointers replaced by the existing scalar kernels; its root cause remains under investigation outside this diff. This PR also intentionally leaves the independent Float2Int8 rounding bug to the companion PR #5023. Combining both fixes passes all 29,120 direct cases across the eight VLEN/VL-policy configurations. This is simulator evidence, not physical-board or pretrained-model validation; the PR remains a draft and makes no performance claim.
Module
CPU / RISC-V RVV
Type
Checklist
[Module:Type] DescriptionformatShared iOS CI failure
The iOS job fails on a pre-existing Xcode project reference to the absent
express/module/MoEModule.cpp. The RVV changes do not modify that project or file. The independent build fix is #5026, which passes the same iOS framework build script locally on Xcode 26.6 / iPhoneOS SDK 26.5. The failed runs on this PR have not been marked as passing; they need a new build with the project fix included.