DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

2026-07-09

AI Summary

This work addresses the inefficiency of existing unified multimodal large models, which generate full images during reasoning, leading to redundant visual tokens and difficulty in capturing critical state changes. To overcome this, the authors propose DeltaV, a novel framework that introduces an incremental visual update paradigm by predicting only compact visual differences between reasoning steps and integrating them with historical visual states for efficient multimodal reasoning. Key innovations include a Temporal Similarity-based Routing Mechanism (TSIM Router) that dynamically allocates update tokens, a large-scale interleaved structured reasoning dataset named StructCoT, and an efficient visual token compression technique. Experiments demonstrate that DeltaV reduces newly generated visual tokens by 55.6% on average while improving multimodal reasoning performance by 3.3%; notably, DeltaV-2B outperforms comparable open-source models by 8.4% and 5.9% on in-domain and out-of-domain benchmarks, respectively.

Abstract

Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large-scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual tokens by 55.6\% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3\% over full-image generation. Trained with StructCoT and large-scale multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4\% on in-domain multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9\% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.

Problem

• Unified Large Multimodal Models
• visual state generation
• visual-token redundancy
• reasoning-critical state transitions
• multimodal reasoning

Innovation

• visual state updates
• unified large multimodal models
• temporal similarity routing
• compact token generation
• multimodal reasoning