Vision-Guided MPPI for Agile Drone Racing: Navigating Arbitrary Gate Poses via Neural Signed Distance Fields

2026-03-07

AI Summary

This work proposes a fully onboard vision-guided optimal control framework for autonomous drone racing that enables high-speed navigation through unknown gates without relying on precomputed trajectories or explicit pose estimation. The core innovation is Gate-SDF, a neural signed distance field implicitly learned from raw depth images, which provides continuous geometric guidance and obstacle avoidance cues. Integrated into an MPPI controller, Gate-SDF facilitates real-time trajectory optimization and achieves zero-shot generalization to arbitrary gate poses—a first in neural SDF–based drone navigation. The approach demonstrates exceptional robustness under severe gate displacements, rotations, occlusions, and sensor noise. Extensive simulations and real-world experiments validate the system’s ability to stably and rapidly complete racing tasks in highly dynamic and uncertain environments.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Learning & Optimization for CV

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Autonomous drone racing requires the tight coupling of perception, planning, and control under extreme agility. However, recent approaches typically rely on precomputed spatial reference trajectories or explicit 6-DoF gate pose estimation, rendering them brittle to spatial perturbations, unmodeled track changes, and sensor noise. Conversely, end-to-end learning policies frequently overfit to specific track layouts and struggle with zero-shot generalization. To address these fundamental limitations, we propose a fully onboard, vision guided optimal control framework that enables reference-free agile flight through arbitrarily placed and oriented gates. Central to our approach is Gate-SDF, a novel, implicitly learned neural signed distance field. Gate-SDF directly processes raw, noisy depth images to predict a continuous spatial field that provides both collision repulsion and active geometric guidance toward the valid traversal area. We seamlessly integrate this representation into a sampling-based Model Predictive Path Integral (MPPI) controller. By fully exploiting GPU parallelism, the framework evaluates these continuous spatial constraints across thousands of simulated trajectory rollouts simultaneously in real time. Furthermore, our formulation inherently maintains spatial consistency, ensuring robust navigation even under severe visual occlusion during aggressive maneuvers. Extensive simulations and real-world experiments demonstrate that the proposed system achieves high-speed agile flight and successfully navigates unseen tracks subject to severe unmodeled gate displacements and orientation perturbations. Videos are available at https://zhaofangguo.github.io/vision_guided_mppi/

Problem

• autonomous drone racing
• spatial perturbations
• zero-shot generalization
• gate pose estimation
• agile flight

Innovation

• Neural Signed Distance Field
• Vision-Guided Control
• Model Predictive Path Integral
• Agile Drone Racing
• Onboard Perception-Planning Integration