Learning to Generate Multiple Objects from Dense and Occluded Layouts

2026-07-03

AI Summary

Existing text-to-image diffusion models struggle to accurately generate the correct number of objects in densely occluded scenes, often suffering from instance merging due to attention confusion and the lack of supervision over non-visible regions. To address this, this work proposes a layout-aware attention bias that encourages consistent token grouping within object regions while suppressing cross-instance information leakage. Additionally, a visibility-weighted balanced loss is introduced for non-visible regions, calibrated by occlusion severity. The authors also construct OverlapDepth-45K, the first benchmark for dense overlapping scenes featuring amodal annotations. Experimental results demonstrate that the proposed approach significantly improves object counting accuracy, effectively mitigates instance merging, and maintains high-fidelity image generation.

Abstract

Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL

Problem

• instance ownership collapse
• dense scenes
• occluded objects
• object count accuracy
• text-to-image generation

Innovation

• layout-aware attention
• amodal-balanced loss
• instance ownership collapse
• dense object generation
• occlusion handling