Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Camera

2024-11-07

IEEE Robotics and Automation Letters

AI Summary

Autonomous obstacle avoidance for monocular optical-flow-driven quadrotors remains challenging due to the lack of differentiable, real-time motion estimation and robust generalization under unseen dense environments. Method: This paper proposes an end-to-end differentiable learning framework comprising (i) a lightweight differentiable optical flow simulator tightly coupled with a simplified quadrotor dynamics model for first-order gradient optimization; (ii) a center-flow attention mechanism to selectively enhance responses to critical ego-motion cues; and (iii) an action-guided active perception strategy to improve cross-scenario robustness. Contribution/Results: Trained exclusively in a minimal simulation environment—without real-world data or domain randomization—the method achieves agile collision-free flight at speeds up to 6 m/s in previously unseen dense scenes. It is successfully deployed on an FPV racing drone platform. Both simulation and physical experiments validate its effectiveness and strong sim-to-real transfer capability.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Optical flow captures the motion of pixels in an image sequence over time, providing information about movement, depth, and environmental structure. Flying insects utilize this information to navigate and avoid obstacles, allowing them to execute highly agile maneuvers even in complex environments. Despite its potential, autonomous flying robots have yet to fully leverage this motion information to achieve comparable levels of agility and robustness. Challenges of control from optical flow include extracting accurate optical flow at high speeds, handling noisy estimation, and ensuring robust performance in complex environments. To address these challenges, we propose a novel end-to-end system for quadrotor obstacle avoidance using monocular optical flow. We develop an efficient differentiable simulator coupled with a simplified quadrotor model, allowing our policy to be trained directly through first-order gradient optimization. Additionally, we introduce a central flow attention mechanism and an action-guided active sensing strategy that enhances the policy's focus on task-relevant optical flow observations to enable more responsive decision-making during flight. Our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system is validated both in simulation and the real world using an FPV racing drone. Despite being trained in a simple environment in simulation, our system demonstrates agile and robust flight in various unknown, cluttered environments in the real world at speeds of up to 6m/s.

Problem

• Enabling agile obstacle avoidance for drones using optical flow
• Overcoming noisy optical flow estimation in high-speed flight
• Ensuring robust performance in complex, unknown environments

Innovation

• End-to-end system using monocular optical flow
• Differentiable simulator with gradient optimization
• Central flow attention for responsive decision-making