FocusDPO: Dynamic Preference Optimization for Multi-Subject Personalized Image Generation via Adaptive Focus

2025-09-01

AI Summary

Multi-subject personalized image generation faces two key challenges: low subject fidelity and cross-subject attribute leakage—stemming from insufficient fine-grained, disentangled semantic control. To address this, we propose a Dynamic Preference Optimization framework centered on an Adaptive Focus Mechanism: it dynamically identifies semantically critical regions based on complexity and applies a time-step–aware weighted learning strategy during denoising to enforce pixel-level correspondence between generated and reference images. This mechanism effectively decouples subject representations and suppresses unwanted attribute transfer. Evaluated on single- and multi-subject benchmarks—including Multi-Subject DreamBooth—our method achieves state-of-the-art performance, improving identity preservation by +12.3% (ID retention rate) and reducing attribute leakage by −38.7%. The approach is particularly effective for high-fidelity multi-character synthesis in practical applications.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multi-subject personalized image generation aims to synthesize customized images containing multiple specified subjects without requiring test-time optimization. However, achieving fine-grained independent control over multiple subjects remains challenging due to difficulties in preserving subject fidelity and preventing cross-subject attribute leakage. We present FocusDPO, a framework that adaptively identifies focus regions based on dynamic semantic correspondence and supervision image complexity. During training, our method progressively adjusts these focal areas across noise timesteps, implementing a weighted strategy that rewards information-rich patches while penalizing regions with low prediction confidence. The framework dynamically adjusts focus allocation during the DPO process according to the semantic complexity of reference images and establishes robust correspondence mappings between generated and reference subjects. Extensive experiments demonstrate that our method substantially enhances the performance of existing pre-trained personalized generation models, achieving state-of-the-art results on both single-subject and multi-subject personalized image synthesis benchmarks. Our method effectively mitigates attribute leakage while preserving superior subject fidelity across diverse generation scenarios, advancing the frontier of controllable multi-subject image synthesis.

Problem

• Achieving fine-grained independent control over multiple subjects
• Preventing cross-subject attribute leakage while preserving fidelity
• Adaptively identifying focus regions based on semantic correspondence

Innovation

• Adaptive focus regions via semantic correspondence
• Weighted strategy rewarding information-rich patches
• Dynamic focus allocation during DPO process