Add Mac metal MLX inference - #50

Open

Ma-Dan wants to merge 16 commits into

Robbyant:mainfrom

Commits on Apr 26, 2026

Add Mac metal MLX inference

Dan
authored and
Dan
committed
d1b19be

Commits on Apr 27, 2026

Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed
8dbd3da

Commits on Apr 29, 2026

Fix float16 scipy error; cap k_special growth

Dan
authored and
Dan
committed
17bf6ec
Speed up _bilinear: precompute flat gather indices, cache as MLX arrays

Dan
authored and
Dan
committed
1e74aa3
Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed
d71796a
Add mx.compile steady-state fast path to CausalAttentionMLX

Dan
authored and
Dan
committed
d88afdc

Commits on Apr 30, 2026

Reduce per-frame overhead: eliminate k_cat temp, cache RoPE, drop eval sync points

Dan
authored and
Dan
committed
9496110
Remove redundant mx.eval(patch_tokens) sync; add steady-state bench tools

Dan
authored and
Dan
committed
434526a
Remove mx.eval(agg_list) mid-forward sync; add bench_e2e.py

Dan
authored and
Dan
committed
b22fd95
Update demo.py help text: document scale-frames as speed lever, fix kv-sliding-window token count

Dan
authored and
Dan
committed
245c570
Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed
32790da
Fix tokens_per_frame to use actual image dimensions (was assuming square)

Dan
authored and
Dan
committed
bf4221d
Correct scale-frames help text; add CLI args to bench_e2e.py; fix tokens_per_frame for non-square images

Dan
authored and
Dan
committed
e9733ce
Update help text: document sw+sf interaction, add combined speedup examples

Dan
authored and
Dan
committed
8a28c36

Commits on May 1, 2026

Add mlx optional dependency extra and guard import with clear error

Dan
authored and
Dan
committed
47155db
Add mlx_infer README

Dan
authored and
Dan
committed
52edc60