InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing

2026-03-10

AI Summary

This work addresses the inherent trade-off in existing unified multimodal models between semantic understanding and generation, which struggle to efficiently balance comprehension, reasoning, generation, and editing. To overcome this limitation, the authors propose InternVL-U, a lightweight unified model with only 4 billion parameters. InternVL-U decouples visual representations from modality-specific modules, employs a unified context modeling framework, and integrates an MMDiT-based visual generation head with chain-of-thought (CoT)-guided synthetic data of high semantic density. Despite its compact size, the model achieves state-of-the-art performance across diverse generation and editing tasks, outperforming baselines with over three times more parameters—such as the 14B-parameter BAGEL—while maintaining strong multimodal understanding and reasoning capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Unified multimodal models (UMMs) that integrate understanding, reasoning, generation, and editing face inherent trade-offs between maintaining strong semantic comprehension and acquiring powerful generation capabilities. In this report, we present InternVL-U, a lightweight 4B-parameter UMM that democratizes these capabilities within a unified framework. Guided by the principles of unified contextual modeling and modality-specific modular design with decoupled visual representations, InternVL-U integrates a state-of-the-art Multimodal Large Language Model (MLLM) with a specialized MMDiT-based visual generation head. To further bridge the gap between aesthetic generation and high-level intelligence, we construct a comprehensive data synthesis pipeline targeting high-semantic-density tasks, such as text rendering and scientific reasoning, under a reasoning-centric paradigm that leverages Chain-of-Thought (CoT) to better align abstract user intent with fine-grained visual generation details. Extensive experiments demonstrate that InternVL-U achieves a superior performance - efficiency balance. Despite using only 4B parameters, it consistently outperforms unified baseline models with over 3x larger scales such as BAGEL (14B) on various generation and editing tasks, while retaining strong multimodal understanding and reasoning capabilities.

Problem

• Unified Multimodal Models
• Semantic Comprehension
• Generation Capabilities
• Multimodal Understanding
• Reasoning

Innovation

• Unified Multimodal Model
• Modality-Specific Modular Design
• MMDiT-based Visual Generation
• Chain-of-Thought Reasoning
• High-Semantic-Density Data Synthesis