ROVER: A Multi-Season Dataset for Visual SLAM

2024-12-03

arXiv.org

AI Summary

Visual SLAM suffers significant robustness degradation in semi-structured natural environments—such as parks—due to seasonal variation, dense vegetation, and extreme illumination changes, particularly under low-light and high-vegetation conditions prevalent in summer and autumn. To address this, we introduce the first outdoor SLAM benchmark dataset covering all four seasons, day–night cycles, and diverse illumination conditions, comprising 39 multi-sensor sequences (monocular, stereo, RGB-D, and IMU). We are the first to systematically incorporate seasonality, vegetation density, and extreme lighting as core evaluation dimensions. Our unified benchmarking platform reveals fundamental failure modes of state-of-the-art methods under challenging conditions—primarily feature scarcity and scale drift. Experiments show that stereo-inertial and RGB-D approaches perform well under ideal lighting, yet all methods exhibit dramatic trajectory error increases during complex summer–autumn scenarios. The dataset, evaluation code, and analysis results are fully open-sourced.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Robust SLAM is a crucial enabler for autonomous navigation in natural, semi-structured environments such as parks and gardens. However, these environments present unique challenges for SLAM due to frequent seasonal changes, varying light conditions, and dense vegetation. These factors often degrade the performance of visual SLAM algorithms originally developed for structured urban environments. To address this gap, we present ROVER, a comprehensive benchmark dataset tailored for evaluating visual SLAM algorithms under diverse environmental conditions and spatial configurations. We captured the dataset with a robotic platform equipped with monocular, stereo, and RGBD cameras, as well as inertial sensors. It covers 39 recordings across five outdoor locations, collected through all seasons and various lighting scenarios, i.e., day, dusk, and night with and without external lighting. With this novel dataset, we evaluate several traditional and deep learning-based SLAM methods and study their performance in diverse challenging conditions. The results demonstrate that while stereo-inertial and RGBD configurations generally perform better under favorable lighting and moderate vegetation, most SLAM systems perform poorly in low-light and high-vegetation scenarios, particularly during summer and autumn. Our analysis highlights the need for improved adaptability in visual SLAM algorithms for outdoor applications, as current systems struggle with dynamic environmental factors affecting scale, feature extraction, and trajectory consistency. This dataset provides a solid foundation for advancing visual SLAM research in real-world, semi-structured environments, fostering the development of more resilient SLAM systems for long-term outdoor localization and mapping. The dataset and the code of the benchmark are available under https://iis-esslingen.github.io/rover.

Problem

• Evaluating visual SLAM in seasonal, varying light conditions
• Addressing performance degradation in dense vegetation environments
• Improving SLAM adaptability for dynamic outdoor factors

Innovation

• Multi-sensor robotic platform for data collection
• Season-spanning outdoor visual SLAM dataset
• Benchmark for diverse environmental conditions