SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

2026-05-12

AI Summary

This work addresses the fragmentation in current vision-language models, which treat understanding and generation as disjoint tasks, leading to architectural inconsistency and misaligned representations that hinder the emergence of native multimodal intelligence. To overcome this, we propose NEO-unify, a novel architecture that natively unifies understanding and generation at the levels of architecture, training, and inference, instantiated in both an 8B dense and a 30B sparse mixture-of-experts variant. Leveraging a mixture-of-experts mechanism, refined data preprocessing, and a joint pretraining–post-training strategy, NEO-unify supports arbitrary-to-image (X2I) synthesis and interleaved text–image generation. Experiments demonstrate that it matches or surpasses state-of-the-art specialized models across diverse tasks—including textual understanding, vision–language perception, knowledge reasoning, agent decision-making, and spatial intelligence—while achieving exceptional semantic coherence and visual fidelity in image generation, with promising potential for extension to vision–language–action systems and world models.

Abstract

Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.

Problem

• multimodal understanding
• multimodal generation
• vision-language models
• unified architecture
• representation alignment

Innovation

• unified multimodal paradigm
• NEO-unify architecture
• vision-language understanding and generation
• mixture-of-experts VLM
• native multimodal intelligence