TokenAR: Multiple Subject Generation via Autoregressive Token-level enhancement

2025-10-17

AI Summary

To address reference identity confusion in autoregressive multi-reference image generation, this paper proposes InstructAR—a token-level enhanced autoregressive framework. It explicitly disentangles subject-specific feature representations via token-index embeddings, instruction token injection, and identity-token decoupling. Further, it improves identity consistency and background fidelity through clustered embeddings, external prior injection, and decoupled learning. Evaluated on the newly constructed large-scale multi-subject dataset InstructAR, the method achieves state-of-the-art performance in identity preservation, background reconstruction quality, and generation diversity. Both code and dataset are publicly released.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Autoregressive Model (AR) has shown remarkable success in conditional image generation. However, these approaches for multiple reference generation struggle with decoupling different reference identities. In this work, we propose the TokenAR framework, specifically focused on a simple but effective token-level enhancement mechanism to address reference identity confusion problem. Such token-level enhancement consists of three parts, 1). Token Index Embedding clusters the tokens index for better representing the same reference images; 2). Instruct Token Injection plays as a role of extra visual feature container to inject detailed and complementary priors for reference tokens; 3). The identity-token disentanglement strategy (ITD) explicitly guides the token representations toward independently representing the features of each identity.This token-enhancement framework significantly augments the capabilities of existing AR based methods in conditional image generation, enabling good identity consistency while preserving high quality background reconstruction. Driven by the goal of high-quality and high-diversity in multi-subject generation, we introduce the InstructAR Dataset, the first open-source, large-scale, multi-reference input, open domain image generation dataset that includes 28K training pairs, each example has two reference subjects, a relative prompt and a background with mask annotation, curated for multiple reference image generation training and evaluating. Comprehensive experiments validate that our approach surpasses current state-of-the-art models in multiple reference image generation task. The implementation code and datasets will be made publicly. Codes are available, see https://github.com/lyrig/TokenAR

Problem

• Addresses identity confusion in autoregressive multi-subject image generation
• Enhances token-level representation for distinct identity preservation
• Improves identity consistency while maintaining background reconstruction quality

Innovation

• Token Index Embedding clusters tokens for reference images
• Instruct Token Injection adds visual features to tokens
• Identity-token disentanglement guides independent identity representation