DeepSeek V4.1 Flash on three M5 Max Macs: native weights, 35.6 tok/s, and current limits

[Jackten]

I've been testing DeepSeek V4.1 Flash on three M5 Max Macs with 128 GiB per worker, using a custom MLX runtime and Thunderbolt/RDMA. The current build is usable for local agent work, with 35.6 output tokens/s on a warmed, single-request test.

This builds on Pipe Network’s DeepSeek V4.1 MLX port, with cluster, native-kernel and serving work on top.

The setup preserves the checkpoint's routed MXFP4 weights, dense MXFP8 weights, BF16 output head, all 40 target layers, and all experts. It uses native DSpark speculation with three complete draft stages. The MLX base is dfe17bafb, with local patches. Model revision: dba1be0a40aa45a94ad051997016db3960a90277.

Measurement	Result
Output generation, fixed 384-token test	31.94 tok/s on the first run; 35.70 and 35.49 on two warmed runs
Earlier hosted DeepSeek reference, same fixture and output cap	150.8 tok/s
First token after a fresh 130,797-token prompt	613.28 s locally; 3.87 s hosted
First token on the repeated local prompt with prefix reuse	1.75 s

These are different measurements: output throughput, fresh-prompt latency, and cached-prompt latency. The hosted speed reference was recorded earlier. The local output path still needs about a 4.24x speedup to match that reference; 35.6 tok/s is a measured result, not a hardware ceiling.

For a small quality check, I froze 108 questions sampled across the public BIG-Bench-Hard tasks, the prompts, and a strict answer/JSON grader before running both systems. Both scored 72/108, with four cases favoring each side and seven output-budget truncations on each. This was a custom sample/protocol, and the tie does not establish broad quality equivalence.

The API has also been exercised with tools, native images, streaming, reasoning, prefix reuse, cancellation, and recovery. OpenClaw and Hermes can use it as a local fallback. Fresh long prompts remain the biggest practical latency problem, and the configured context is 131,072 tokens.

I'm pausing performance tuning at this checkpoint. I'd be interested in other people's measured native-weight V4.1 decode results on Apple Silicon, especially end-to-end results for speculative verification and expert placement. What hardware, draft configuration, prompt/output lengths, and timing boundaries are you using? Input-speed results would also be useful if they separate fresh prefill from cache hits and describe any approximation.

Disclosure: the implementation, evaluation scripts, and this write-up were developed with AI assistance. This is a field report from a custom build; it is not a stock MLX benchmark or a reproducible public release yet.

[Jackten]

The three-Mac source kit is now public: Jackten/deepseek-v41-mlx-three-mac.

It includes the native-weight runtime, complete three-stage DSpark, pinned MLX patch and build script, checkpoint preparation, portable launch configuration, and setup instructions. Upstream attribution and licenses are retained.

The new measured change is long-prompt startup: five matched ~31K-token samples gave median TTFT 96.66 s → 56.68 s with suffix scheduling and the original float32 cache. A fresh warm 384-token bicycle test measured 35.44 tok/s; across six matched warm decode cells, differences from the original profile stayed within 1.11%, with identical output token sequences. This is a startup improvement; it does not establish a decode speedup. The 39-sample dataset and method retain warmup rules and the excluded pair.

The pinned backend was rebuilt and the offline suite passed 784 tests plus 113 subtests. This is an experimental source release from one qualified deployment; the new portable wrapper still needs end-to-end qualification on a fresh cluster. Reproductions and measured fixes are welcome. Implementation, tests and this update were AI-assisted; the linked results were executed and checked.