Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark

2026-04-30

AI Summary

This work addresses the challenges of autonomous driving on unstructured roads at night, where visible-light perception is unreliable, infrared datasets are scarce, and single-frame methods suffer from inter-frame inconsistency. To this end, the authors introduce IRON, the first large-scale temporal infrared dataset tailored for all-weather unstructured road scenarios, and propose IRONet—a flow-free temporal free-space detection framework. IRONet leverages a memory attention mechanism and a mask decoder to effectively aggregate historical contextual information across frames. The approach significantly enhances temporal consistency and nighttime robustness, achieving 82.93% IoU and 90.66% F1-score on IRON, outperforming existing methods. Furthermore, it demonstrates strong generalization capability to RGB modalities when evaluated on the ORFD and Rellis datasets.

Abstract

Off-road nighttime autonomous driving suffers from unreliable visible-light perception, making infrared modality crucial for accurate freespace detection. However, progress remains limited due to the scarcity of annotated infrared off-road datasets and the inter-frame inconsistencies inherent to current single-frame methods. To address these gaps, we present the IRON dataset, which, to our knowledge, is the first large-scale infrared dataset for off-road temporal freespace detection under all-day conditions, with strong support for nighttime perception. The dataset comprises 24,314 densely annotated infrared images with synchronized RGB images in diverse scenes and different light conditions. Building upon this dataset, we propose IRONet, a novel flow-free framework for temporal freespace detection that addresses inter-frame inconsistencies by aggregating historical context via a memory-attention mechanism and a carefully designed mask decoder. On our IRON dataset, IRONet achieves state-of-the-art performance, reaching 82.93%(+1.19%) IoU and 90.66%(+0.71%) F1 score at real-time inference. Remarkably, IRONet also exhibits robust generalization to RGB modalities on ORFD and Rellis datasets. Overall, our work establishes a foundation for reliable all-day off-road autonomous driving and future research in infrared temporal perception. The code and IRON dataset are available at https://github.com/wsnbws/IRON.

Problem

• off-road driving
• infrared perception
• freespace detection
• all-day autonomy
• temporal consistency

Innovation

• multispectral dataset
• temporal freespace detection
• infrared perception
• memory-attention mechanism
• off-road autonomous driving