ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detection

2025-10-17

AI Summary

Existing generative data augmentation methods for object detection suffer from content-location misalignment and semantic leakage, often requiring complex post-processing or extensive fine-tuning. This paper introduces ReCon—the first region-controllable diffusion-based augmentation framework tailored for object detection. Its core innovations are: (1) integrating pretrained perceptual model feedback directly into the diffusion sampling process to dynamically correct erroneous regions; and (2) introducing region-aligned cross-attention to explicitly enforce semantic and spatial consistency between generated content and user-specified bounding boxes. Experiments across multiple datasets, backbone architectures, and training scales demonstrate that ReCon consistently improves detection performance. It significantly enhances both semantic fidelity and spatial accuracy of generated samples while improving model trainability—without requiring additional fine-tuning.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The scale and quality of datasets are crucial for training robust perception models. However, obtaining large-scale annotated data is both costly and time-consuming. Generative models have emerged as a powerful tool for data augmentation by synthesizing samples that adhere to desired distributions. However, current generative approaches often rely on complex post-processing or extensive fine-tuning on massive datasets to achieve satisfactory results, and they remain prone to content-position mismatches and semantic leakage. To overcome these limitations, we introduce ReCon, a novel augmentation framework that enhances the capacity of structure-controllable generative models for object detection. ReCon integrates region-guided rectification into the diffusion sampling process, using feedback from a pre-trained perception model to rectify misgenerated regions within diffusion sampling process. We further propose region-aligned cross-attention to enforce spatial-semantic alignment between image regions and their textual cues, thereby improving both semantic consistency and overall image fidelity. Extensive experiments demonstrate that ReCon substantially improve the quality and trainability of generated data, achieving consistent performance gains across various datasets, backbone architectures, and data scales. Our code is available at https://github.com/haoweiz23/ReCon .

Problem

• Addresses content-position mismatches in generative data augmentation
• Overcomes semantic leakage issues in synthesized object detection samples
• Enhances spatial-semantic alignment between image regions and text cues

Innovation

• Region-guided rectification in diffusion sampling
• Region-aligned cross-attention for spatial-semantic alignment
• Feedback from pre-trained perception model