Qwen opens Qwen3.8-Flash-Next, a 6B-active MoE previewing Qwen4 architecture
- Qwen released Qwen3.8-Flash-Next weights, a multimodal MoE and early public preview of the architecture planned for the Qwen4 model family.
- The model has 125B main parameters plus 51B N-gram embedding parameters, but activates 6B parameters per token; Qwen says training used about one-ninth the cost of Qwen3.7-Plus.
- Its GDN + QSA attention combines Gated DeltaNet history compression with Qwen Sparse Attention, which uses a lightweight indexer to select context at micro-block granularity.
- Gated Residual expands the residual stream into four branches with dynamic read and write gates, while N-gram embeddings use local context and can be offloaded to host memory with asynchronous prefetching.
- Qwen lists native 262,144-token context, extension to 1 million tokens with YaRN, and a hosted Qwen3.8-Flash price of $0.16 per million input tokens and $0.47 per million output tokens.
Hacker News 의견들
This looks better than DeepSeek V4 Flash on the published results.
The benchmarks are seriously impressive for an MoE with only 6B active parameters.
I get the fatigue with daily Qwen posts, but this is a new architecture previewing Qwen4, not a routine checkpoint. Qwen says it trained at about one-ninth the cost of Qwen3.7-Plus while beating it across the board.
This matters to people running local models on AMD AI and Apple hardware. A release can affect what I can run for work over the next few months more than most blog posts do.
I ran Qwen3.6 35B A3B on four L4s and want to see whether this works on six. vLLM has no TP6, so I may need TP2 plus MTP, and I'll have to test what is actually viable.
The interesting part is the Qwen4 architecture preview and the direct comparison with Qwen3.8-27B. N-gram embeddings appear to be a first public release here, and a 6B-active MoE should be cheap and fast to serve if the runtime support arrives.
I did not expect it to beat Qwen3.8-27B so cleanly. People are talking about roughly 30 tokens per second on a MacBook, which is a strange place for self-hosted models to be.
I dislike the Opus comparison. This may match larger models on some benchmarks, but it does not have their world knowledge.
For a hosted comparison, GPT 5.6 Luna gets 67% on DeepSWE versus 59% here, at $0.20 per million input tokens and $1.20 output tokens. Qwen lists $0.16 and $0.47.
I care more about token efficiency now than raw benchmark scores for Chinese models. I want to know how much test-time reasoning it burns to get these results.
A lot of apparent overthinking is extra test-time compute. Even meaningless thinking tokens can improve results because the model gets more passes to refine activations, so the visible chain of thought is not the whole mechanism.
I have found Qwen3.8 can overthink on xhigh, especially with one-shot open-ended prompts. With clear requirements or an unambiguous path, it can be concise and has not increased my task completion time in practice.
Nothing seemed to run this easily at first. Mainline llama.cpp and vLLM lacked support for the new architecture.
Qwen has a vLLM recipe and a specially provided vLLM release. I would expect upstream llama.cpp and vLLM support to take a few days.
Qwen's listed API price is not below DeepSeek Flash if the comparison is DeepSeek 0723. Qwen lists $0.16 per million input tokens and $0.47 output, while DeepSeek lists $0.03 and $0.075.
The Unsloth GGUF download is a 72.5GB 1-bit quant, not a 4-bit model. That is larger than I expected, and it is less appealing for Strix Halo if a usable 4-bit quant will not fit.