Qwen3-Next-80B-A3B-Instruct
Qwen3-Next-80B-A3B-Instruct is the first in the Qwen3-Next series, featuring groundbreaking architectural innovations. It uses Hybrid Attention combining Gated DeltaNet and Gated Attention for efficient ultra-long context modeling, High-Sparsity MoE with 512 experts (10 activated + 1 shared) achieving extreme low activation ratio, and Multi-Token Prediction for improved performance and faster inference. With 80B total parameters and only 3B activated, it outperforms Qwen3-32B-Base with 10% training cost and 10x throughput for 32K+ contexts. The model performs on par with Qwen3-235B-A22B-Instruct-2507 while excelling at ultra-long-context tasks up to 256K tokens (extensible to 1M with YaRN). Architecture: 48 layers, 15T training tokens, hybrid layout of 12*(3*(Gated DeltaNet->MoE)->(Gated Attention->MoE)).
Benchmark results
| Benchmark | Score | Tags | Source |
|---|---|---|---|
| Aider-Polyglot | 49.8% | self-reported llm-stats | link → |
| AIME 2025 | 69.5% | self-reported llm-stats | link → |
| Arena-Hard v2 | 82.7% | self-reported llm-stats | link → |
| BFCL-v3 | 70.3% | self-reported llm-stats | link → |
| Creative Writing v3 | 85.3 | self-reported llm-stats | link → |
| GPQA | 72.9% | self-reported llm-stats | link → |
| HMMT25 | 54.1% | self-reported llm-stats | link → |
| IFEval | 87.6% | self-reported llm-stats | link → |
| Include | 78.9% | self-reported llm-stats | link → |
| LiveBench 20241125 | 75.8% | self-reported llm-stats | link → |
| LiveCodeBench v6 | 56.6% | self-reported llm-stats | link → |
| MMLU-Pro | 80.6% | self-reported llm-stats | link → |
| MMLU-ProX | 76.7% | self-reported llm-stats | link → |
| MMLU-Redux | 90.9% | self-reported llm-stats | link → |
| Multi-IF | 75.8% | self-reported llm-stats | link → |
| MultiPL-E | 87.8% | self-reported llm-stats | link → |
| PolyMATH | 45.9% | self-reported llm-stats | link → |
| SuperGPQA | 58.8% | self-reported llm-stats | link → |
| TAU-bench Airline | 44.0% | self-reported llm-stats | link → |
| TAU-bench Retail | 60.9% | self-reported llm-stats | link → |
| Tau2 Airline | 45.5% | self-reported llm-stats | link → |
| Tau2 Retail | 57.3% | self-reported llm-stats | link → |
| Tau2 Telecom | 13.2% | self-reported llm-stats | link → |
| WritingBench | 87.3% | self-reported llm-stats | link → |