Enable half-precision complex Metal FFT kernels - #3981
Open
PhysicistJohn wants to merge 5 commits into
Open
Conversation
Replace the float-specific Metal complex struct with complex_t<T> while retaining complex64_t as an alias and preserving existing kernel entry points. Generalize scalar and cross-complex conversions, arithmetic, and limits so half and bfloat lane types retain packed two-lane storage without implicit float promotion.
Template C2C FFT arithmetic and I/O over their scalar lane while retaining the existing static entry points and packed real paths. Make the C2C four-step second pass use the same generic storage path so future scalar lanes preserve the existing indexing and scaling behavior. Derive twiddle phase evaluation precision from the scalar lane and Metal's pi constant instead of naming float in generic FFT code, while preserving current code generation.
Teach FFTValueTraits to load and store complex_t<T> while preserving lane-typed vector arithmetic. Add a compile-only Metal regression for half and bfloat16 adapters without adding production kernel instantiations or changing the shipped metallib.
Instantiate C2C Stockham, Rader, Bluestein, and four-step kernels for packed complex_t<half> storage. Move convolution and four-step inverse scaling ahead of reduced-precision accumulation so representable results do not overflow, and compute reciprocals before narrowing to the storage lane.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Depends on #3969, #3970, and the
Metal FFT storage adapter.
The isolated one-commit implementation diff is
PhysicistJohn/mlx#7.
Summary
Enable packed
complex_t<half>C2C FFT kernels for Stockham, Rader,Bluestein, and four-step plans. The patch adds the raw Metal backend capability;
public dtype and host-dispatch changes are outside this diff.
Comparable GPU-library support is established in
NVIDIA cuFFT, which supports
half-precision C2C transforms, and
AMD rocFFT,
which supports FP16 real and complex transforms.
The implementation commit changes five FFT files with 60 insertions and 33
deletions.
Implementation
complex32_t = complex_t<half>and assert its four-byte, two-lanelayout.
sizes.
accumulation, avoiding overflow of finite half-complex results in temporary
buffers.
Correctness
Candidate FP32 output is bit-identical to untouched
mainFP32 for therepresentative Stockham, Rader, Bluestein, and four-step cases below.
A separate half-complex sweep covered 39 lengths from 2 through 1,048,576.
Every forward and inverse result was finite. Worst normalized RMSE was 0.2585%
forward and 0.4101% round trip; the 1,048,576-point round-trip error was 0.2685%.
Performance against untouched main
Apple M5 Max; untouched
mainatfb5133e1; implementation commit at3c63352d. Each variant used the same source-level algorithm harness, FFTplans, dispatch geometry, 4,194,304 complex values, input samples, five
warmups, 21 timed samples, and 50 transforms per sample. Six runs covered all
six execution orders of main FP32, candidate FP32, and candidate half. Values
below are the median of the six run medians.
Forward
Inverse
The candidate FP32 control measured 0.992x-1.065x the original FP32 throughput
across these eight cases while remaining bit-identical. A second six-run Rader
and Bluestein measurement reproduced their inverse results.
Packed complex storage and external I/O are four bytes per value instead of
eight. These are direct kernel measurements, not public
mx.fftAPI results.Validation
-Wall -Wextra -Werror.integration tree.
symbols.
git diff --check.Binary impact
fft.airmlx.metallib