Subject-Consistent and Pose-Diverse Text-to-Image Generation

2025-07-11

AI Summary

Addressing the challenge of simultaneously preserving subject identity consistency and enabling pose/compositional diversity in text-to-image (T2I) generation, this paper introduces CoDi—a two-stage diffusion control framework. In the early denoising stage, a pose-aware optimal transport mechanism enables identity-preserving subject transfer; in the later stage, saliency-guided feature selection and explicit enhancement of critical identity features jointly disentangle and optimize consistency and diversity. CoDi requires no additional training and supports parameter-free inference-time optimization. Quantitative and qualitative evaluations demonstrate that CoDi outperforms state-of-the-art methods across key metrics—including subject identity consistency, pose diversity, and prompt fidelity—particularly in complex visual narrative tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Subject-consistent generation (SCG)-aiming to maintain a consistent subject identity across diverse scenes-remains a challenge for text-to-image (T2I) models. Existing training-free SCG methods often achieve consistency at the cost of layout and pose diversity, hindering expressive visual storytelling. To address the limitation, we propose subject-Consistent and pose-Diverse T2I framework, dubbed as CoDi, that enables consistent subject generation with diverse pose and layout. Motivated by the progressive nature of diffusion, where coarse structures emerge early and fine details are refined later, CoDi adopts a two-stage strategy: Identity Transport (IT) and Identity Refinement (IR). IT operates in the early denoising steps, using optimal transport to transfer identity features to each target image in a pose-aware manner. This promotes subject consistency while preserving pose diversity. IR is applied in the later denoising steps, selecting the most salient identity features to further refine subject details. Extensive qualitative and quantitative results on subject consistency, pose diversity, and prompt fidelity demonstrate that CoDi achieves both better visual perception and stronger performance across all metrics. The code is provided in https://github.com/NJU-PCALab/CoDi.

Problem

• Maintain consistent subject identity across diverse scenes
• Achieve pose and layout diversity in generated images
• Balance subject consistency with expressive visual storytelling

Innovation

• Two-stage diffusion strategy for consistency and diversity
• Identity Transport for pose-aware feature transfer
• Identity Refinement for salient feature enhancement