GLM-5.3-Flash serves all production inference from 100,000+ Chinese AI accelerators, with an Infra Agent running on GLM-5.3 doing much of the build
- Z.ai built a production-grade inference service for GLM-5.3-Flash from scratch on a cluster of more than 100,000 Chinese-made AI accelerators, and all production inference for the model runs on that system.
- Much of the build was done by an Infra Agent powered by GLM-5.3, which carried the model from initial adaptation to production launch in under two weeks and tripled end-to-end throughput over the initial baseline.
- Tested anonymously as Ox-Alpha on OpenCode and OpenRouter, GLM-5.3-Flash became the most-used model on both platforms within a week of launch, processing more than 62 trillion tokens in six days.
- Memory optimizations including intra-node tensor parallelism for linear attention and the LM Head, ReplaySSM, W8A8 quantization, INT8/FP8/BF16 mixed-precision cache quantization, Layer Split, and an Encode-Prefill-Decode disaggregated architecture improved serving performance about 3x, with hardware utilization and per-token cost comparable to mainstream NVIDIA GPUs.
- Z.ai reports that the Infra Agent's effectiveness depends more on continuous feedback traceable to a specific cause than on code generation, because metrics like a 30% TTFT increase tell the agent results got worse without saying which layer is responsible.
Hacker News opinions
Most of us kinda guessed this when they decided to hand out 100 trillion tokens for free.
Very few people grasp how much of an asteroid level event this is for western AI labs. China has cheap abundant power, and now they make their own inference chips, which was supposed to be the chokepoint. Their models can be 6 months behind the frontier, but most people don't need frontier models, small ones are more than enough.
Wish labs like Mistral would build their own inference chips or partner with established and new chip makers, or companies like Oxide.
I was gonna ask how people found their coding plans, then realized they massively ramped up prices. Middle plan is about 80 bucks a month now, didn't it used to be like 20? They must have hit hard scaling limits to hike it that fast.
Yeah it went from a great deal to unviable compared to other providers. They need a healthy middle ground.
Hard to say, nobody advertises real token limits since they're probably adaptive. The 80 dollar plan is still roughly 80 dollars of quota. On the API they charge a bit more than the cheapest GLM-5.3-Flash providers, not enough to explain a big price gap.
I paid 360 for the annual Max plan and average about 1BN tokens a day on GLM-5.3. That was clearly unsustainable for them, and they dropped the package.
I'm on a legacy plan and my renewal would put me on the watered down one, though the 50% legacy discount might still make it worth it. Their plans were initially comically subsidized, more advert for an upstart than commercial pricing.
Their plans are still worth it if you use their models. Max gives roughly 1,100 USD of GLM-5.3 or 260 USD of flash per month for 168. Flash is slow though, I get about 40 tok/s on 5.3 and 30 on flash.
Sure, aside from the infrastructure they got by illegally routing millions of paying customers' requests through Anthropic's Opus 4.8 in a distillation attack.
What exactly is illegal about it?
That's a canard. Anthropic and OpenAI encrypt thinking token outputs, Chinese labs don't. Everyone is probably using open-weight models in their synthetic data pipelines anyway, and it's way easier to distill from logits than from hard tokens.
Even if true, they're stealing from thieves. Anthropic paid out a copyright settlement covering basically every author on the planet, so there's no reason to respect terms they impose.
If you think they're bottlenecked on training data here, that's absurd. You build simulation environments with RLVR scoring to train reasoning models, so why wouldn't Zhipu use GLM to build its own RL environments?
This left me with one question: WTF is GLM? No idea what z.ai is either, so it's a bit presumptuous to assume everyone knows their product.
It's their own blog, and the first sentence already says GLM is a model. z.ai is a well known Chinese lab, GLM is probably the most popular model outside Anthropic and OpenAI. You're just out of the loop.
It's the top open-weights LLM in the world right now.
Different angle on the same model: full GLM-5.3 (744B MoE, 4-bit experts, 434 GB on disk) runs on a single MacBook Pro M5 Max with 128 GB by streaming experts off NVMe. One drive gives about 2 tok/s, four striped drives hit 3.5 tok/s byte-identical, and an unpublished patch does 4.2.
Seems unusably slow. Is that for short context only?