PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation

2026-07-07

AI Summary

Current text-to-image generation models struggle to align with users’ implicit visual preferences. To address this limitation, this work proposes the first user-profile-driven evaluation benchmark that integrates psychological and demographic dimensions. The authors establish a collaborative data generation pipeline involving real users and AI agents and introduce a multidimensional framework for evaluating personalized image generation. Leveraging this benchmark, they systematically assess state-of-the-art personalization methods, uncovering critical shortcomings in their ability to align with individual user preferences. The findings highlight key challenges and offer concrete directions for future research in personalized generative modeling.

Abstract

Recent text-to-image models such as DALLE-3 excel at following diverse prompts yet remain blind to individual aesthetic preferences. We study personalized image generation, where models must align outputs with a user's implicit visual preferences based on a few historically preferred images and a short prompt. To this end, we introduce PIPBench, the first profile-inclusive benchmark for evaluating personalized image generation. We further propose a novel data construction pipeline that leverages psychological and demographic profiling dimensions for both real-user data collection and scalable agent-based data generation. Using PIPBench, we conduct a thorough evaluation of representative line of methods. Our experiments reveal key limitations in existing methods, suggesting new challenges and opportunities for personalized text-to-image synthesis. Project page: https://wuyuhang05.github.io/PIPBench/

Problem

• personalized image generation
• aesthetic preferences
• text-to-image synthesis
• user profiling
• image evaluation

Innovation

• personalized image generation
• profile-inclusive benchmark
• agent-based data generation
• aesthetic preference modeling
• text-to-image synthesis