AIR: Adaptive Interleaved Reasoning with Code in MLLMs

2026-06-22

AI Summary

This work addresses the limitations of existing tool-calling approaches in visual perception tasks, which rely on handcrafted heuristic rules and struggle with complex numerical reasoning. To overcome this, the authors propose an adaptive code-interleaved reasoning framework based on reinforcement learning, enabling multimodal large language models to dynamically invoke computational tools. The approach introduces three key innovations: a two-stage cold-start data construction pipeline, a reinforcement learning data filtering strategy, and a population-constrained reward mechanism. Evaluated on standard benchmarks, the method achieves an average performance gain of 6.1 percentage points, improves accuracy on interleaved reasoning samples by 9.9 percentage points, and attains a tool invocation success rate exceeding 95%.

Abstract

Following the paradigm shift initiated by OpenAI o3, interleaved reasoning with code to enhance multimodal large language models (MLLMs) has become a pivotal research frontier. The existing literature focuses primarily on tool-use within vision-perception tasks. However, such approaches typically rely on predefined heuristics for visual manipulation and are inherently incapable of addressing numerical computation problems due to their exclusive focus on visual operations. This paper empowers MLLMs with adaptive interleaved reasoning capabilities through extended reinforcement learning training on code-augmented complex numerical computation tasks. To this end, we propose a comprehensive three-component solution consisting of: a two-stage cold-start data construction pipeline, data filtering strategies for RL dataset curation, and an adaptive tool-invocation strategy leveraging a group-constrained reward function for interleaved reasoning trajectories. Extensive experiments demonstrate that after Reinforcement Learning training with the group-constrained reward function, performance improves by an average of 6.1 percentage points (pp) on evaluation benchmarks. Specifically, the accuracy for interleaved reasoning samples increases by 9.9 pp, and the overall success rate of tool-use exceeds 95%. Our data and code are available at: https://github.com/CongHan0808/AIR.git.

Problem

• interleaved reasoning
• multimodal large language models
• numerical computation
• tool-use
• code-augmented reasoning

Innovation

• Adaptive Interleaved Reasoning
• Code-Augmented Reasoning
• Reinforcement Learning
• Multimodal Large Language Models
• Tool-Use