M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models

2026-05-10

AI Summary

Existing agents struggle to effectively integrate mathematical reasoning with general-purpose reasoning in multitask learning, often resulting in unstable behavior and limited performance gains. This work proposes the M2A paradigm, which, for the first time, decouples these two reasoning modalities in parameter space. Specifically, mathematical reasoning capabilities are injected into the null-space directions of the agent’s model via task vectors, thereby preserving the original functionality without interference. Additionally, a tunable coefficient is introduced to control the depth of reasoning. Notably, this approach requires no additional training and boosts the pass rate of Qwen3-8B on SWE-Bench Verified from 44.0% to 51.2%, substantially enhancing both reasoning depth and stability.

Abstract

While reasoning has become a central capability of large language models (LLMs), the reasoning patterns required for different scenarios are often misaligned. Mathematical reasoning typically relies on intrinsic logic to solve closed-world problems in a single response, whereas agentic reasoning requires not only internal reasoning but also multi-turn interaction with external environments, interleaving thought and action. This misalignment prevents mathematical and agentic reasoning from effectively benefiting from each other, often yielding unstable reasoning behavior and only limited performance gains under multi-task learning. In this paper, we propose M2A, a novel paradigm that synergizes mathematical and agentic reasoning via model merging. To avoid overfitting to superficial reasoning patterns under joint training, M2A operates directly in parameter space: it identifies the feature subspace critical for agent behavior, and merges the mathematical reasoning task vector only along its null space, thereby injecting reasoning capability along directions that do not perturb agent behavior. Unlike SFT or RL, M2A requires no additional gradient-update and exposes the merging coefficient as a simple knob for controlling reasoning length. Experiments in a challenging real-world coding agent setting show that our method effectively extends agentic reasoning depth and delivers substantial performance improvements. Applied to a fine-tuned Qwen3-8B, M2A improves its SWE-Bench Verified resolved rate from 44.0% to 51.2% without retraining the model. Code is available at https://github.com/laplucky/M2A.git.

Problem

• mathematical reasoning
• agentic reasoning
• reasoning alignment
• large language models
• multi-task learning

Innovation

• model merging
• mathematical reasoning
• agentic reasoning
• parameter space
• null space injection