MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation

2026-07-01

AI Summary

This work addresses the challenges in multi-subject personalized image generation—such as subject omission, appearance distortion, and interaction mismatch—and the absence of effective evaluation metrics. The authors propose MIBE, a unified framework comprising a Multi-subject Interaction Benchmark (MIB) and a lightweight diagnostic evaluator (MIE), establishing the first standardized benchmark for multi-subject interactions. By decoupling data mechanisms to span diverse relationships and scene complexities, and leveraging 60k vision-language model silver labels alongside 4k double-blind human gold-standard annotations, the evaluator is trained with a dual-head ranking and diagnostic objective. MIE achieves a 0.922 overall pairwise accuracy on the gold-standard set, with 0.982 and 0.884 for seen and unseen generators, respectively, significantly outperforming baselines like CLIP and DINO, while demonstrating high alignment with human preferences and strong generalization across generators.

Abstract

Multi-subject personalized image generation requires the precise rendering of all requested reference identities and their specified interactions based on a guiding prompt. However, state-of-the-art models still struggle with this process, frequently omitting subjects, failing to preserve reference appearances, or misattributing interactions. Furthermore, existing metrics designed primarily for single-subject fidelity cannot reliably capture these errors, suffering severe degradation in ranking separability and failing to align with human preference as the subject count increases. To address this gap, we introduce Multi-subject Interaction Benchmark and Evaluator (MIBE), a unified framework comprising a Multi-subject Interaction Benchmark (MIB) and a Multi-subject Interaction Evaluator (MIE). MIB systematically covers diverse relation types and scene complexities through a decoupled data regime. This consists of a 60K-pair VLM-labeled Silver Set for scalable metric training and a 4K-pair double-blind Human Evaluation Gold Set covering a diverse range of state-of-the-art generators, with the Silver Set reaching 95.1% cross-VLM preference agreement. To demonstrate the utility of this benchmark, we present MIE, a lightweight, reference-conditioned evaluator trained exclusively on the Silver Set with a dual-head ranking and diagnosis objective. MIE exhibits strong cross-generator generalization on the Gold Set, achieving 0.922 overall pairwise accuracy against human preference, including 0.982 on seen generators and 0.884 on unseen generators. By outperforming a broad spectrum of baseline metrics, including CLIP and DINO variants, MIE demonstrates that diagnostic supervision can preserve ranking separability and human alignment where traditional evaluators collapse.

Problem

• multi-subject personalized image generation
• subject interaction
• fidelity evaluation
• human preference alignment
• ranking separability

Innovation

• multi-subject personalized generation
• interaction benchmark
• reference-conditioned evaluator
• human-aligned metric
• diagnostic supervision