Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch

2025-12-02

AI Summary

Existing methods decouple image manipulation from network search, rely on costly reinforcement learning, and lack planning grounded in real tool-execution trajectories. This work proposes a dynamic interleaved reasoning mechanism that unifies multimodal planning, visual manipulation, and deep search for the first time—enabling long-horizon intelligent behavior without reinforcement learning under a supervised fine-tuning framework. Our approach leverages a high-quality dataset of <30K planning-execution aligned trajectories and introduces stepwise consistency filtering to ensure reliable reasoning. Evaluated on MMSearch and FVQA, it achieves 66.1 and 67.2, respectively—surpassing Gemini 2.5 Flash across all 11 benchmark metrics. Moreover, it supports complex cross-modal task solving requiring over ten sequential tool invocations.

Technology Category

• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Despite recent progress in multimodal agentic systems, existing approaches often treat image manipulation and web search as disjoint capabilities, rely heavily on costly reinforcement learning, and lack planning grounded in real tool-execution traces. To address these limitations, we present Skywork-R1V4, a 30B (A3B) parameter multimodal agentic model that unifies multimodal planning, active image manipulation ("thinking with images"), deep multimodal search, and, most critically, interleaved reasoning that dynamically alternates between visual operations and external knowledge retrieval. Trained solely via supervised fine-tuning on fewer than 30,000 high-quality, planning-execution-consistent trajectories and validated through stepwise consistency filtering, Skywork-R1V4 achieves state-of-the-art results across perception and multimodal search benchmarks: it scores 66.1 on MMSearch and 67.2 on FVQA, surpassing Gemini 2.5 Flash on all 11 metrics. Skywork-R1V4 exhibits emergent long-horizon reasoning at inference time, successfully orchestrating more than 10 tool calls to solve complex, multi-step tasks. Our results demonstrate that sophisticated agentic multimodal intelligence can be achieved through carefully curated supervised learning alone, without any reliance on reinforcement learning.

Problem

• Unifies multimodal planning and image manipulation with deep search.
• Addresses disjoint capabilities and heavy reliance on reinforcement learning.
• Enables interleaved reasoning between visual operations and knowledge retrieval.

Innovation

• Unifies multimodal planning with active image manipulation
• Integrates deep multimodal search and interleaved visual-knowledge reasoning
• Achieves state-of-the-art performance via supervised fine-tuning only