360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents

2026-08-09

AI Summary

This work addresses the limitations of existing outdoor navigation benchmarks, which lack sufficient realism and environmental complexity to effectively evaluate embodied agents’ exploration capabilities in real-world urban settings. To bridge this gap, the authors construct a high-fidelity, city-scale virtual environment using 602 segments of 360-degree street-view footage from Akihabara, Tokyo, and introduce 175 human-annotated tasks spanning three core competencies: environmental understanding, path reasoning, and spatial reasoning. This benchmark significantly narrows the sim-to-real gap, revealing that even the strongest current model, Gemini 2.5 Flash, achieves only a 17.1% task success rate—substantially lower than the human performance of 77.3%—thereby underscoring the substantial challenges that remain in urban-scale embodied navigation.

Abstract

We present 360CityArena, a benchmark for evaluating the urban exploration capabilities of embodied agents within a photorealistic environment constructed from 360-degree videos. Existing outdoor benchmarks either lack sufficient photorealism or complexity, resulting in a considerable gap from real-world urban environments. 360CityArena is built on a realistic reconstruction of the Akihabara district in Tokyo, Japan, using 602 360-degree video segments covering 85 streets, and consists of 175 meticulously human-crafted tasks. It encompasses three task categories: Environment Understanding, Path Reasoning, and Spatial Reasoning, covering fundamental abilities required for urban exploration, such as localization, landmark search, path planning, and relational spatial reasoning, thereby enabling comprehensive evaluation in realistic urban scenes. Our evaluation using state-of-the-art LMM-based agents shows that even the strongest model, Gemini 2.5 Flash, performs far below human level (human: 77.3% vs. Gemini 2.5 Flash: 17.1%), revealing substantial challenges that remain in city-scale embodied navigation and reasoning. 360CityArena provides a necessary and challenging testbed for photorealistic urban-district navigation and spatial reasoning.

Problem

• embodied agents
• urban navigation
• photorealistic environment
• spatial reasoning
• benchmark

Innovation

• photorealistic urban navigation
• 360-degree video reconstruction
• embodied agent benchmarking
• spatial reasoning
• city-scale simulation