DiffusionGemma converts Gemma 4 into a 1,500-token/s diffusion language model on one H100
- DiffusionGemma generates about 1,500 output tokens per second on one NVIDIA H100, averaging roughly 20 generated tokens per forward pass across its evaluation suite.
- Instead of autoregressive one-token decoding, the model iteratively denoises 256-token blocks in parallel, avoiding the sequential decoding bottleneck.
- The team converted Gemma 4, a mixture-of-experts model with 3.8B active and 25.2B total parameters, by fine-tuning it rather than training a new model from scratch.
- Its two-stage conversion uses supervised bidirectional denoising, then reinforcement learning and sampler distillation, with under 10% of the original autoregressive model's training-token budget.
- DiffusionGemma retains thinking mode, multimodal input, and long-context support, while still supporting autoregressive generation with minor reported performance loss.
Hacker News opinions
I find diffusion text models fascinating, though the idea of starting with noise across a response and filling it in still breaks my brain. I am probably misunderstanding something fundamental.
DiffusionGemma denoises several "canvases," so it can keep reasoning separate from a final-reply canvas. It is the most interesting local LLM approach to me because one GPU can handle a response much faster. I get about 670 tok/s on a Pro 6000, depending on the text.
I found a visual guide that made the design much easier to understand. They converted an existing Gemma 4 26B A4B MoE checkpoint into a denoiser by using logits for every token, rather than training from scratch. I hope other open models get similar diffusion conversions.
I see this more as a competing approach for improving compute utilization at low batch sizes than as a direct route to JEPA or AGI.
The speed results look appealing, but I want to know whether diffusion models can close the accuracy gap with autoregressive models. Bidirectional reasoning and self-correction might even turn into an advantage.
I reimplemented it for macOS and like the model a lot. It reasons fairly well and is easy to bend to a specific use. I get about 15 tok/s on M3-class hardware, and 20 to 30 tok/s with a suitable draft model, though I have not combined them into a net speedup yet.