Support W8A8 hy4-preview inference on PPU 810E - #477
Open
ZenithJoyH wants to merge 13 commits into
Open
ZenithJoyH wants to merge 13 commits into
ZenithJoyH wants to merge 13 commits into
Conversation
Refresh and finalize the HY4 preview model, loader, dispatch, and runtime integration. Preserve the T-Head fallback when the native bundle is unavailable.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Redispatch the dynamic full-width candidate sort to native ATen so enabling FlagGems topk does not compile a new specialized kernel at every decode step.
Collaborator
|
support in which vendor? |
Contributor
Author
I have test GPQA on PPU，But the key operator is called by using flaggems. |
ZenithJoyH
changed the title
ZenithJoyH
changed the title
ZenithJoyH
force-pushed
the
support-hy4-preview
branch
from
September 10, 2026 07:55
2693074 to
236d98b
Compare
Preserve explicit per-token decode lengths while reusing the common FlagGems and reference row-wise top-k capability inside the HY4 BF16 indexer backend.
ZenithJoyH
force-pushed
the
support-hy4-preview
branch
from
September 10, 2026 08:23
236d98b to
4f9630c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Category
Others
PR Type
User Experience
Description
Support hy4 preview inference on PPU 810E
Related Issues
Changes
Testing
test GPQA on PPU
Checklist