AVOID: The Adverse Visual Conditions Dataset with Obstacles for Driving Scene Understanding

2025-12-29

AI Summary

Existing road datasets lack multimodal annotations of real-scale small obstacles under adverse weather and lighting conditions, limiting the robustness of real-time detection. To address this, we introduce AVOID—the first driving dataset explicitly designed for real-time obstacle detection under challenging conditions including rain, fog, and low illumination. Each frame provides synchronized, co-registered RGB images, semantic maps, depth maps, raw and semantic LiDAR point clouds, and trajectory points. We establish a novel paradigm for multi-sensor synchronization and high-precision calibration within a unified visual domain, enabling a joint multi-task benchmark for semantic segmentation, depth estimation, and trajectory prediction. Constructed via high-fidelity simulation, AVOID integrates YOLO-based detectors and a custom multi-task U-Net architecture. Experiments demonstrate significantly improved obstacle detection robustness; the multi-task model outperforms single-task baselines by +3.2% in mIoU, −11.7% in depth RMSE, and notably enhances trajectory prediction accuracy.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Understanding road scenes for visual perception remains crucial for intelligent self-driving cars. In particular, it is desirable to detect unexpected small road hazards reliably in real-time, especially under varying adverse conditions (e.g., weather and daylight). However, existing road driving datasets provide large-scale images acquired in either normal or adverse scenarios only, and often do not contain the road obstacles captured in the same visual domain as for the other classes. To address this, we introduce a new dataset called AVOID, the Adverse Visual Conditions Dataset, for real-time obstacle detection collected in a simulated environment. AVOID consists of a large set of unexpected road obstacles located along each path captured under various weather and time conditions. Each image is coupled with the corresponding semantic and depth maps, raw and semantic LiDAR data, and waypoints, thereby supporting most visual perception tasks. We benchmark the results on high-performing real-time networks for the obstacle detection task, and also propose and conduct ablation studies using a comprehensive multi-task network for semantic segmentation, depth and waypoint prediction tasks.

Problem

• Detects unexpected small road hazards in real-time
• Addresses lack of adverse condition datasets with obstacles
• Provides multi-sensor data for comprehensive visual perception tasks

Innovation

• Simulated environment dataset with adverse conditions
• Multi-sensor data integration for perception tasks
• Benchmarking real-time networks with ablation studies