Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification

2026-06-16

AI Summary

This work addresses the limitation of existing unified multimodal models, which rely on separate visual tokenizers that decouple the representation spaces for understanding and generation. To bridge this gap, the authors propose UniAR, a framework that, for the first time, enables end-to-end autoregressive modeling of both tasks within a shared context using a single discrete visual tokenizer. Key innovations include a multi-level feature-fused visual encoder, lookup-free bit quantization, a parallel bit prediction mechanism that substantially compresses visual sequence length, and a diffusion-based visual decoder. Combined with large-scale pretraining, supervised fine-tuning, and reinforcement learning, UniAR achieves state-of-the-art performance in image generation and editing while maintaining competitive results on multimodal understanding benchmarks.

Abstract

Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.

Problem

• Unified Multimodal Modeling
• Visual Tokenizer
• Autoregressive Modeling
• Representation Space
• Multimodal Understanding

Innovation

• unified multimodal modeling
• shared visual tokenizer
• autoregressive generation
• bitwise quantization
• parallel bitwise prediction