PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes

2026-08-16

AI Summary

This study addresses the challenges of spatial orientation control, attribute leakage, and artifact generation in multi-object text-to-image synthesis by proposing a lightweight 2.5D controllable generation framework. The method introduces a standardized OrientLayout dataset alongside a context-aware dual-stream representation and parallel masking architecture. By integrating MM-DiT with sparse spatial angular anchors, the framework achieves precise layout manipulation. Experimental results demonstrate that this approach significantly outperforms state-of-the-art methods in spatial precision, orientation accuracy, and multi-object visual fidelity. Consequently, it effectively resolves persistent issues regarding generation consistency and controllability within complex scenes, offering a robust solution for fine-grained compositional image generation.

Abstract

While Text-to-Image (T2I) diffusion models have achieved remarkable success, precise spatial and orientational control in multi-object scenes remains a persistent challenge. Existing methods either rely on computationally expensive dense 3D maps or suffer from severe attribute leakage and "cut-and-paste" artifacts. To address these limitations, we propose PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation. Instead of dense spatial maps, it establishes precise spatial-angular anchors using an efficient condition layout: individual object captions, 2D bounding boxes, and 3D angles. To resolve the generative trade-off between strict instance isolation and global coherence, we introduce a Context-Aware Dual-Stream Representation. By injecting local object tokens and relation-enriched scene tokens into the visual stream of modern MM-DiT architectures via parallel masked and unmasked pathways, PoseAdapter eliminates attribute leakage while preserving natural inter-object relationships and scene-level coherence. To support this paradigm, we construct OrientLayout, a high-quality dataset featuring standardized 2.5D annotations and instance-level decoupled semantics. Extensive experiments demonstrate that PoseAdapter outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity. Code and dataset will be available at https://github.com/cyf23/PoseAdapter.

Problem

• Controllable Image Generation
• Multi-Object Scenes
• Spatial Control
• Attribute Leakage
• 2.5D Generation

Innovation

• PoseAdapter
• 2.5D Controllable Generation
• Context-Aware Dual-Stream Representation
• OrientLayout
• Attribute Leakage