Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments

2026-07-08

AI Summary

This work addresses the challenge of efficient dynamic obstacle avoidance for mobile robots in unstructured outdoor environments, where large-scale training data and reliable simulation are often unavailable. The authors propose a training-free, purely vision-based approach that leverages the pre-trained monocular depth model UniDepth to generate dense depth maps. By integrating SuperPoint/SuperGlue keypoint tracking with bundle adjustment, the method reconstructs 3D motion from real-world RGB video sequences and computes per-point time-to-collision (TTC) estimates to determine safe navigation directions within the ground plane. Requiring only 74 seconds of real-world data for parameter tuning, the approach entirely circumvents sim-to-real transfer issues and exhibits high data efficiency, behavioral interpretability, and strong generalization across diverse obstacle types. Evaluated on the M3ED dataset, it achieves 0.49 precision and 0.38 recall for frames with TTC < 1 second, correctly generates valid avoidance directions in 84% of true-positive cases, and provides effective early warnings for 20 out of 22 obstacle categories.

Abstract

Dynamic obstacle avoidance in unstructured outdoor environments remains a critical challenge for autonomous mobile robots, particularly when large-scale robot-specific training data and simulation-based policies are impractical. We present a data-efficient, interpretable method for vision-based dynamic obstacle avoidance that operates entirely on real-world data, avoiding the sim-to-real transfer problem inherent in simulation-trained policies. Our approach leverages UniDepth, a large pretrained monocular depth estimation model, to produce dense depth maps from RGB video without requiring stereo cameras or LiDAR at inference time. Dynamic obstacle avoidance is achieved by extending the SuperPoint and SuperGlue feature correspondence pipeline to track keypoints across long frame sequences, projecting their 2D pixel-space positions into 3D using camera intrinsics and predicted depth, running bundle adjustment initialized from these 3D keypoints, and computing per-keypoint time-to-collision (TTC). A 2D motion primitive in the ground plane is then selected to move the robot away from the closest point of approach of the minimum-TTC keypoint. Evaluated on real-world data from the M3ED dataset, our pipeline achieves a precision of 0.49 and a recall of 0.38 in identifying frames with a ground truth TTC below 1 second, and correctly generates the evasive motion direction in 84\% of true positive detections. Crucially, it detects at least one frame with TTC less than 1 second for 20 out of 22 unique physical obstacles present in our test sequences. Unlike end-to-end learned methods that demand thousands of hours of robot-specific training data, our approach eliminates model training entirely, requiring only 74 seconds of data for hyperparameter tuning. This demonstrates exceptional data efficiency while preserving interpretable and generalizable behavior across diverse obstacle types.

Problem

• dynamic obstacle avoidance
• unstructured environments
• time-to-collision
• autonomous mobile robots
• sim-to-real transfer

Innovation

• time-to-collision
• pretrained vision models
• monocular depth estimation
• dynamic obstacle avoidance
• data-efficient robotics