DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement

2026-05-24

AI Summary

This work addresses the challenge of representation interference in unified multimodal models, where conflicting objectives between generation and understanding tasks hinder joint optimization. To resolve this, the authors propose DIVA, a framework that explicitly disentangles visual representations within a unified architecture into shared and task-specific components. By leveraging mutual information estimation, DIVA enables complementary information flow and decoupled training between understanding and generation branches. This approach reveals, for the first time, the complementary internal structure of representations in unified multimodal models. Combined with a self-improving post-training strategy, DIVA significantly enhances collaborative multimodal performance—achieving gains of 7.82% on visual understanding tasks and 8.46% on generation tasks—outperforming all existing baselines comprehensively.

Abstract

Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lies in inductive biases induced by distinct supervision signals: generation branch prefers high-fidelity, fine-grained representations capable of reconstruction, while the understanding favours semantically discriminative embeddings that remain invariant to task-irrelevant factors. Consequently, optimizing these complementary but non-equivalent objectives within a monolithic backbone leads to mutual impairment instead of enhancement. In this paper, we first analyze the root cause of this interference in unified backbones and reveal a complementary structure in their internal representations. Motivated by the observation, we propose DIVA, a self-improved post-training framework that transforms the representation divergence into interior synergy. By explicitly factorizing the visual representation into shared and unique components based on two complementary information flow, DIVA enables both the understanding and generation branches to achieve beneficial transferring while preserving the integrity of unique information from cross-flow interference via mutual information estimation. Despite its generality, our method consistently achieves improvements across visual understanding (+7.82%) and generation (+8.46%). The official code is available at: https://github.com/Jayyy-H/DIVA.

Problem

• Unified Multimodal Models
• Representation Divergence
• Understanding-Generation Interference
• Inductive Bias
• Multimodal Representation

Innovation

• representation divergence
• unified multimodal models
• mutual information estimation
• factorized representation
• post-training framework