Z.ai open-sources 320B-parameter GLM-5.3-Flash for million-token video inputs
- Z.ai open-sourced GLM-5.3-Flash, a multimodal 320-billion-parameter model that can process up to 1 million tokens, including video inputs.
- The mixture-of-experts model activates 18 billion parameters per prompt and uses sparse plus linear attention to reduce memory and hardware overhead on long inputs.
- The article says linear attention avoids the RAM growth associated with standard softmax attention when prompt length doubles, while processing sequences in O(n) rather than O(n²) complexity.
- Z.ai trained the model on 30 trillion tokens and used mHC technology to prevent gradient distortion during model reconfiguration.
- OpenRouter added GLM-5.3-Flash within 48 hours and lists it at $0.15 per million input tokens, according to the article.