Object-centric Self-improving Preference Optimization for Text-to-Image Generation

2025-05-28

AI Summary

Multimodal large language models (MLLMs) exhibit insufficient fine-grained visual understanding in text-to-image generation, particularly regarding object-level layout, attribute consistency, and relational reasoning. Method: We propose an object-centric self-optimizing preference learning framework that requires no external data or models. It introduces novel object-level prompt perturbation and compositional prompt density enhancement, coupled with VQA-driven automatic scoring and filtering to autonomously construct high-quality, highly discriminative object-level contrastive preference pairs—effectively eliminating ambiguous and imbalanced samples. Contribution/Results: Through multi-step end-to-end preference optimization, our method significantly improves object localization accuracy, attribute fidelity, and relational modeling capability. It achieves state-of-the-art performance across three compositional text-to-image benchmarks, demonstrating the effectiveness and scalability of the data-self-generating preference optimization paradigm.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent advancements in Multimodal Large Language Models (MLLMs) have significantly improved both image understanding and generation capabilities. Despite these improvements, MLLMs still struggle with fine-grained visual comprehension, particularly in text-to-image generation tasks. While preference optimization methods have been explored to address these limitations in image understanding tasks, their application to image generation remains largely underexplored. To address this gap, we propose an Object-centric Self-improving Preference Optimization (OSPO) framework designed for text-to-image generation by MLLMs. OSPO leverages the intrinsic reasoning abilities of MLLMs without requiring any external datasets or models. OSPO emphasizes the importance of high-quality preference pair data, which is critical for effective preference optimization. To achieve this, it introduces a self-improving mechanism that autonomously constructs object-level contrastive preference pairs through object-centric prompt perturbation, densification and VQA scoring. This process eliminates ambiguous or disproportionate variations commonly found in naively generated preference pairs, thereby enhancing the effectiveness of preference optimization. We validate OSPO on three representative compositional text-to-image benchmarks, demonstrating substantial performance gains over baseline models.

Problem

• Improving fine-grained visual comprehension in text-to-image generation
• Addressing lack of preference optimization for image generation tasks
• Enhancing object-level contrastive preference pairs autonomously

Innovation

• Object-centric Self-improving Preference Optimization framework
• Autonomous object-level contrastive preference pairs construction
• No external datasets or models required