Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection

2025-04-23

AI Summary

Existing generative data augmentation methods for autonomous driving suffer from spatial layout distortion of inserted objects, primarily due to neglecting physical plausibility and semantic constraints—resulting in unrealistic object placements. Method: This paper proposes a scene-aware probabilistic placement modeling framework that jointly encodes geometric structure, semantic relationships, and contextual cues by integrating multimodal scene understanding, probabilistic graphical models, and generative image inpainting. A learnable spatial prior is further introduced to predict physically feasible and semantically consistent placement regions aligned with real-world driving scenes. Contribution/Results: Evaluated on two autonomous driving object detection benchmarks, our method establishes new state-of-the-art performance for generative augmentation, achieving a +1.4 mAP improvement over the previous best approach (a 2.8× gain relative to the runner-up). It also yields significant gains in instance segmentation accuracy.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Generative image models are increasingly being used for training data augmentation in vision tasks. In the context of automotive object detection, methods usually focus on producing augmented frames that look as realistic as possible, for example by replacing real objects with generated ones. Others try to maximize the diversity of augmented frames, for example by pasting lots of generated objects onto existing backgrounds. Both perspectives pay little attention to the locations of objects in the scene. Frame layouts are either reused with little or no modification, or they are random and disregard realism entirely. In this work, we argue that optimal data augmentation should also include realistic augmentation of layouts. We introduce a scene-aware probabilistic location model that predicts where new objects can realistically be placed in an existing scene. By then inpainting objects in these locations with a generative model, we obtain much stronger augmentation performance than existing approaches. We set a new state of the art for generative data augmentation on two automotive object detection tasks, achieving up to $2.8 imes$ higher gains than the best competing approach ($+1.4$ vs. $+0.5$ mAP boost). We also demonstrate significant improvements for instance segmentation.

Problem

• Enhancing realism in automotive object detection data augmentation
• Improving scene-aware object placement for augmented frames
• Maximizing performance gains in generative data augmentation

Innovation

• Scene-aware probabilistic location model predicts realistic object placement
• Generative inpainting enhances augmented object realism in scenes
• Achieves state-of-the-art performance in automotive object detection