Cartesia says its byte-level H-Net matches a tokenized open-source model with 10x less compute and data
- A two-stage H-Net outperforms a matched open-source model while using 10x less compute and data, and a carefully controlled one-stage H-Net gives 3-5x compute efficiency gains purely from learned tokenization, per Cartesia Research.
- H-Nets replace the fixed tokenizer with recursive, data-dependent chunking done inside the model during end-to-end training, instead of BPE chunking applied offline before training starts.
- The authors report in-context tokenization, a behavior they say has not appeared in tokenized language models, and cite ongoing results applying H-Nets to biology and vision as evidence of a general architecture.
- Cartesia frames the work against Sutton's bitter lesson: learn meaning from raw bytes with as few human priors as possible, and calls byte-level language modeling a litmus test for whether general learning methods work at all.
- H-Nets were introduced by the team in 2025 (arXiv 2507.07955); this post is part one of two, with the second covering a wider range of model sizes to explain where the efficiency comes from.
Hacker News opinions
That haskell quicksort tokenization example in the post got me feeling pretty good about the whole thing.
It kind of goes against everything NLP optimized for with BPE and redundancy in large data. Put this on a coding task or plain English and it underperforms a tokenizer that exploits multi-char patterns, and it decodes much slower.
With spec decode where it is now, I've seen 32-token spec-decode show up in frontier models, you'd need a very domain-specific use for this to pay off. Massive tabular data maybe, but at that point it's already structured, so just use an LLM with polars or sql or arrow.
The main premise is how general and widely applicable it is. They show it working on images too. One shared tokenizing paradigm across modalities is a big step for consolidating how everything gets preprocessed, which matters if intelligence is going to scale across modalities.
The architecture can use multi-char patterns too, they just aren't hard-coded like BPE is. And you can still use speculative decoding. Whether it ends up better or worse is something you have to actually measure.