SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks

2026-06-15

AI Summary

This work addresses the lack of a unified benchmark for evaluating visual navigation models on urban sidewalks, which hinders quantitative assessment under complex layouts, dynamic pedestrian interactions, and long-horizon tasks. To this end, we introduce SidewalkBench—the first high-fidelity simulation benchmark tailored to this domain—built on NVIDIA Isaac Sim and integrating both procedurally generated and real-world scanned environments. It features an event-driven model of dynamic pedestrian behavior and supports multi-scale, reproducible, and standardized evaluation. We systematically assess nine state-of-the-art models across 330 unit tests, 800 interactive scenarios, and 105 long-horizon tasks, revealing critical limitations in handling pedestrian interactions and maintaining robustness over extended trajectories. Our experiments further demonstrate that synthetic data augmentation significantly enhances model performance.

Abstract

Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While recent visual navigation models offer a promising solution, the lack of a unified benchmark hinders quantitative and reproducible evaluation. To bridge this gap, we propose SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks. Built upon NVIDIA Isaac Sim, SidewalkBench brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes. We further populate the scenes with rich, reactive event-based pedestrian behaviors and flexible, efficient animation, enabling standardized model evaluation under realistic real-world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.

Problem

• visual navigation
• urban sidewalks
• benchmarking
• pedestrian interaction
• long-horizon navigation

Innovation

• visual navigation
• urban sidewalk simulation
• GPU-accelerated benchmarking
• reactive pedestrian behavior
• synthetic data scaling