Next-Concept Modeling generates text coarse-to-fine and runs up to 31x faster than the strongest diffusion language model
- Next-Concept Modeling is a two-stage coarse-to-fine scheme: a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting every code in that level in parallel.
- At matched parameter scales NCM gets better generative perplexity than the strongest diffusion language model baseline on LM1B while running 21x faster, is competitive on OpenWebText at 10x faster, and matches that baseline's ROUGE-L on XSum at 31x faster.
- On Sudoku, NCM beats autoregressive and diffusion baselines at every difficulty level with fewer function evaluations.
- The authors motivate the work by the weaknesses of diffusion language models: quality drops fast with fewer sampling steps, the full sequence is processed at every step, and many passes over the dataset are needed to reach good quality.
- The paper is arXiv:2610.14162, submitted 9 Oct 2026 by Hermann Kumbong and eight co-authors including Simran Arora, Azalia Mirhoseini and Christopher Ré.
Hacker News opinions
Somebody with a real cluster needs to train a big coding model on this, because I sure don't have the GPUs for it.
I keep hoping Apple Silicon catches up to Nvidia for inference. It's still more than 5x behind in some cases, prefill especially, and that tile-based mobile GPU design may need a serious redesign before it gets close.
Tried to get it running myself and got nowhere so far.
The clickbait saying NCM obsoletes LLMs is nonsense. LLM just means large and about language, nothing about autoregressive ordering, and NCM is a hierarchical diffusion transformer, basically an evaluation strategy you could bolt onto other architectures. The name is bad too, since 'next' implies an ordering that LLM never claimed.
Half these people think an LLM is whatever another LLM told them it is when they asked it to write a script.
Could tricks from image and video compression and partitioning carry over to this kind of dimensional masking?
It's clever, but a new tokenizer plus a new training method means a lot of retraining. Whether anyone adopts it depends on the frontier performance it actually reaches.
Using × instead of the word 'times' bugs me. One of the first things I was taught is that mathematical notation isn't prose.
What about 'got taught' instead of 'was taught' and the missing comma before 'but'?
Does the 10 in 10× count as mathematical notation too, or is only 'ten times' allowed?
If we can't build hardware fast enough, why do hyperscalers supposedly have 50% of the GPUs and memory they bought sitting in warehouses for lack of data centers? They're hoarding gear they can't even use.