Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning

2026-08-14

AI Summary

This study addresses the efficiency bottleneck caused by the coupling of knowledge storage and reasoning in large language models by proposing Mobius, a novel decoupled architecture. Mobius introduces a globally shared Feed-Forward Network (FFN) as a knowledge memory, coordinated with multiple self-attention reasoners for iterative querying, and employs a hidden state caching mechanism to enable efficient knowledge retrieval and compositional reasoning. Experiments demonstrate that a 7B model trained with 37.4% less data achieves performance comparable to baselines, while a 35B model attains nearly a fourfold increase in end-to-end inference speed without sacrificing accuracy. By effectively facilitating knowledge reuse, this work significantly reduces data requirements and substantially enhances inference efficiency, offering a scalable solution to the storage-reasoning trade-off in modern LLMs.

Abstract

We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory for required knowledge-vectors, while the knowledge is transmitted back to reasoning operators. Through this knowledge-reasoning-separation architecture, Mobius achieves better knowledge compression and reasoning efficiency. Built upon Mobius-v0 architecture: 1) Our 7B model trained-from-scratch achieves similar downstream score as a 7B Transformer baseline with 62.6% of baseline's training data. 2) Our Intern-S2-Mobius, continually-pretrained from Qwen3.5-35B, achieves similar downstream score while delivering nearly 4x end-to-end inference speedup.

Problem

• Foundation Model
• Knowledge Compression
• Reasoning Efficiency
• Compositional Reasoning

Innovation

• Decoupled Knowledge and Reasoning
• Mobius Architecture
• Shared Memory FFN
• Iterative Compositional Reasoning
• Inference Speedup