COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

2026-06-26

AI Summary

Existing unified multimodal models exhibit limitations in fine-grained compositional understanding and controllable generation. This work proposes COMPASS, a novel framework that unifies composition-aware perception and generation within a single system for the first time, leveraging a shared expert token τ_c as an anchor for compositional intent to shift from passive analysis to explicit layout control. Built upon a Mixture-of-Experts (MoE) backbone, the approach introduces lightweight composition experts, expert token distillation, global conditioning via denoising trajectories, and an inference-enhanced compositional annotation strategy. Additionally, the authors curate Comp-11, a large-scale compositional instruction dataset. Experiments demonstrate that COMPASS significantly advances category-level compositional understanding and generates outputs with superior composition consistency and prompt fidelity compared to strong baselines.

Abstract

Composition is a high-level visual intent that governs where subjects are placed and how a scene is organized, yet current unified multimodal models remain unreliable at fine-grained composition recognition and struggle to turn such intent into controllable generation. We present COMPASS, the first unified multimodal framework that grounds composition-intent control in a single system spanning both composition perception and composition-guided generation, with a shared expert token $τ_c$ as the central intent anchor. On the perception side, COMPASS injects composition expertise into an MoE backbone in a minimally invasive manner and distills the inferred intent into $τ_c$. On the generation side, COMPASS reuses $τ_c$ as a global conditioning signal that steers the denoising trajectory, effectively converting passive composition analysis into explicit layout control. To support systematic instruction-following composition learning and evaluation at scale, we construct Comp-11, a large-scale dataset with an 11-class taxonomy and reasoning-augmented annotations. Extensive experiments show that COMPASS substantially improves category-level composition understanding and delivers more composition-consistent, prompt-faithful generation than strong baselines.

Problem

• composition
• multimodal models
• intent grounding
• controllable generation
• composition recognition

Innovation

• composition-intent grounding
• unified multimodal model
• expert token
• composition-guided generation
• MoE backbone