Synthesizing and Identifying Noise Levels in Autonomous Vehicle Camera Radar Datasets

2025-05-01

AI Summary

This paper addresses the degradation of camera-radar fusion data in autonomous driving caused by sensor failures and real-world interference. We propose the first noise modeling, synthesis, and identification framework specifically designed for automotive-grade multimodal data. Methodologically: (1) we develop a quantifiable joint image–point-cloud noise-level synthesis pipeline that generates 11 representative degradation patterns based on physical models; (2) we design a lightweight CNN-RNN hybrid network for noise identification, enhanced with cross-modal registration and a multi-source heterogeneous data co-annotation strategy. Evaluated on 10,086 images and 2,145 radar point-cloud frames, our method achieves a 54.4% noise-type identification accuracy—significantly outperforming baseline approaches—and demonstrates that noise-aware training substantially improves the robustness of downstream 3D detection and tracking.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Detecting and tracking objects is a crucial component of any autonomous navigation method. For the past decades, object detection has yielded promising results using neural networks on various datasets. While many methods focus on performance metrics, few projects focus on improving the robustness of these detection and tracking pipelines, notably to sensor failures. In this paper we attempt to address this issue by creating a realistic synthetic data augmentation pipeline for camera-radar Autonomous Vehicle (AV) datasets. Our goal is to accurately simulate sensor failures and data deterioration due to real-world interferences. We also present our results of a baseline lightweight Noise Recognition neural network trained and tested on our augmented dataset, reaching an overall recognition accuracy of 54.4% on 11 categories across 10086 images and 2145 radar point-clouds.

Problem

• Enhancing robustness of AV object detection to sensor failures
• Creating synthetic data for camera-radar noise simulation
• Developing lightweight noise recognition model for AV datasets

Innovation

• Synthetic data augmentation for AV camera-radar datasets
• Simulating sensor failures and real-world interferences
• Lightweight Noise Recognition neural network