Problem
Research questions and friction points this paper is trying to address.
Embodied Urban Navigation
Long-tail Distribution
Safety-critical Scenarios
Data Collection Cost
Exposing the Long-tail in Embodied Urban Navigation via Scalable Learning from In-the-Wild Videos
🤖 AI Summary
This study addresses the scarcity of urban navigation data and insufficient coverage of long-tail safety scenarios by proposing a scalable learning framework. By automatically annotating trajectory semantics from massive in-the-wild videos and integrating vision-language-action policies with reflective mechanisms, the approach enables knowledge transfer from unstructured video to real-world navigation alongside interpretable planning. The research not only validates the efficacy of video-based knowledge transfer but also systematically reveals and characterizes the intrinsic coherence within the long-tail structure of navigation tasks. Consequently, this work establishes a novel paradigm for enhancing the safety and robustness of autonomous driving in complex urban environments, effectively bridging the gap between large-scale video data and reliable downstream navigation performance.
📝 Abstract
Learning embodied urban navigation policies from real-world data is constrained by the cost of task-specific data collection and the limited coverage of rare yet safety-critical scenarios. To address these challenges, we present a scalable framework for learning point-goal urban navigation from web-scale in-the-wild egocentric videos while systematically exposing its long tail. The framework automatically annotates uncurated web videos with metric trajectories and structured navigation semantics, which are then used to train a vision-language-action policy for interpretable navigation planning. We characterize the long tail based on model performance and the distribution of perception-motion patterns, and employ reflection-based analysis to diagnose recurring failure modes. Experiments on web-video data and real-world urban navigation tasks demonstrate effective knowledge transfer from unconstrained videos and reveal coherent long-tail structures beyond aggregate navigation performance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Scalable Learning
In-the-Wild Videos
Vision-Language-Action Policy
Long-Tail Characterization
Embodied Urban Navigation
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
