Verti-Bench: A General and Scalable Off-Road Mobility Benchmark for Vertically Challenging Terrain

2025-02-17

AI Summary

Existing off-road mobility evaluation lacks standardized, objective benchmarks for extremely rugged and vertical terrains. Method: We introduce the first generalizable, scalable mobility benchmark tailored to high-challenge off-road environments, featuring 100 heterogeneous terrains, 1,000 navigation tasks, and a million-scale multi-physical terrain attribute dataset. It establishes the first dynamic mobility evaluation paradigm for vertical terrain, integrating high-fidelity multi-physics simulation, semantic-geometric coupled terrain modeling, rigid/deformable ground simulation, and a Gym-style reinforcement learning interface. The benchmark further provides failure case repositories and expert demonstrations to enable interactive, rather than static, data-driven assessment. Contribution/Results: Unified evaluation across ten diverse off-road platforms uncovers common failure modes—including rollovers and getting stuck—revealing critical bottlenecks in robust terrain perception and adaptive dynamical control, thereby guiding future research in resilient off-road autonomy.

Technology Category

• Application Domains: Mobility, Driving & Flight
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Recent advancement in off-road autonomy has shown promises in deploying autonomous mobile robots in outdoor off-road environments. Encouraging results have been reported from both simulated and real-world experiments. However, unlike evaluating off-road perception tasks on static datasets, benchmarking off-road mobility still faces significant challenges due to a variety of factors, including variations in vehicle platforms and terrain properties. Furthermore, different vehicle-terrain interactions need to be unfolded during mobility evaluation, which requires the mobility systems to interact with the environments instead of comparing against a pre-collected dataset. In this paper, we present Verti-Bench, a mobility benchmark that focuses on extremely rugged, vertically challenging off-road environments. 100 unique off-road environments and 1000 distinct navigation tasks with millions of off-road terrain properties, including a variety of geometry and semantics, rigid and deformable surfaces, and large natural obstacles, provide standardized and objective evaluation in high-fidelity multi-physics simulation. Verti-Bench is also scalable to various vehicle platforms with different scales and actuation mechanisms. We also provide datasets from expert demonstration, random exploration, failure cases (rolling over and getting stuck), as well as a gym-like interface for reinforcement learning. We use Verti-Bench to benchmark ten off-road mobility systems, present our findings, and identify future off-road mobility research directions.

Problem

• Evaluating off-road mobility in rugged terrains
• Standardizing mobility benchmarks for diverse vehicle platforms
• Simulating complex vehicle-terrain interactions for autonomy

Innovation

• High-fidelity multi-physics simulation
• Scalable to various vehicle platforms
• Gym-like interface for reinforcement learning