Qwen3 Technical Report

2025-05-14

AI Summary

To address the fundamental trade-offs among performance, efficiency, multilingual capability, and dynamic reasoning in large language models, this work introduces the Qwen3 series (0.6B–235B), featuring a novel unified “thinking/non-thinking” dual-mode architecture with user-controllable inference budgeting—enabling runtime adaptive mode switching. We propose a synergistic design integrating Mixture-of-Experts (MoE) and dense architectures, augmented by chain-of-thought knowledge distillation and multilingual joint pretraining, expanding language coverage from 29 to 119 languages while substantially reducing training costs for smaller variants. Qwen3 achieves state-of-the-art results on code, mathematical reasoning, and agent-oriented benchmarks—matching or exceeding the performance of significantly larger MoE models and leading proprietary models. All model weights are released under the Apache 2.0 license, ensuring full openness and reproducibility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities. The Qwen3 series includes models of both dense and Mixture-of-Expert (MoE) architectures, with parameter scales ranging from 0.6 to 235 billion. A key innovation in Qwen3 is the integration of thinking mode (for complex, multi-step reasoning) and non-thinking mode (for rapid, context-driven responses) into a unified framework. This eliminates the need to switch between different models--such as chat-optimized models (e.g., GPT-4o) and dedicated reasoning models (e.g., QwQ-32B)--and enables dynamic mode switching based on user queries or chat templates. Meanwhile, Qwen3 introduces a thinking budget mechanism, allowing users to allocate computational resources adaptively during inference, thereby balancing latency and performance based on task complexity. Moreover, by leveraging the knowledge from the flagship models, we significantly reduce the computational resources required to build smaller-scale models, while ensuring their highly competitive performance. Empirical evaluations demonstrate that Qwen3 achieves state-of-the-art results across diverse benchmarks, including tasks in code generation, mathematical reasoning, agent tasks, etc., competitive against larger MoE models and proprietary models. Compared to its predecessor Qwen2.5, Qwen3 expands multilingual support from 29 to 119 languages and dialects, enhancing global accessibility through improved cross-lingual understanding and generation capabilities. To facilitate reproducibility and community-driven research and development, all Qwen3 models are publicly accessible under Apache 2.0.

Problem

• Advancing performance, efficiency, and multilingual capabilities in LLMs
• Integrating thinking and non-thinking modes for dynamic reasoning
• Reducing computational resources while maintaining competitive performance

Innovation

• Unified thinking and non-thinking modes framework
• Thinking budget mechanism for adaptive computation
• Knowledge transfer to optimize smaller models