01 What happened
The PyTorch team at Hugging Face detailed contributions to PyTorch 2.15's MPS backend, featuring FP8 tensors, 5–6.5× MoE decoding speedups, and dramatic video memory reductions from 270 to 8 GiB.
02 Key details
- PyTorch 2.15 adds torch.float8_e4m3fn tensors on MPS: one byte per value, half storage of FP16 or BF16, with F.scaled_mm supporting both tensorwise and rowwise scaling.
- MoE models decode 5–6.5× faster on M3 Ultra: Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s; LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s.
- Wan2.2 tiled VAE decode peak driver memory drops from ~270 GiB to ~8 GiB; Qwen-Image-2.1 VAE encoder now matches CPU (PSNR 10.06→29.95 dB).
- F.linear uses specialized GEMV kernels for decode; softmax about 8× faster; complex LU factorization; public MPS streams; more native operations.
03 Why it matters
Release fixes performance barriers for on-device AI on Macs. Dedicated Metal kernels make large language and video models more practical.
04 Who it matters to
AI developers.
Original source