Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing

2026-03-09

AI Summary

This work addresses the challenge of modeling non-smooth tasks—such as gate traversal and obstacle avoidance—in high-speed autonomous drone racing, which are difficult to capture with conventional differentiable losses. To this end, we propose DiffRacing, a novel framework that, for the first time, integrates task-oriented geometric priors in the form of vector fields into differentiable policy learning. This enables joint optimization of gate navigation and collision avoidance through continuous gradient signals. Furthermore, we introduce a differentiable Delta action model to bridge the sim-to-real gap in dynamics without requiring explicit system identification. By combining differentiable physics simulation with end-to-end visual policies, our approach achieves higher sample efficiency, faster convergence, and robust high-speed flight performance in both simulated and real-world environments.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Autonomous drone racing in complex environments requires agile, high-speed flight while maintaining reliable obstacle avoidance. Differentiable-physics-based policy learning has recently demonstrated high sample efficiency and remarkable performance across various tasks, including agile drone flight and quadruped locomotion. However, applying such methods to drone racing remains difficult, as key objective like gate traversal are inherently hard to express as smooth, differentiable losses. To address these challenges, we propose DiffRacing, a novel vector field-augmented differentiable policy learning framework. DiffRacing integrates differentiable losses and vector fields into the training process to provide continuous and stable gradient signals, balancing obstacle avoidance and high-speed gate traversal. In addition, a differentiable Delta Action Model compensates for dynamics mismatch, enabling efficient sim-to-real transfer without explicit system identification. Extensive simulation and real-world experiments demonstrate that DiffRacing achieves superior sample efficiency, faster convergence, and robust flight performance, thereby demonstrating that vector fields can augment traditional gradient-based policy learning with a task-specific geometric prior.

Problem

• drone racing
• obstacle avoidance
• gate traversal
• differentiable policy learning
• vector field

Innovation

• vector field
• differentiable policy learning
• drone racing
• sim-to-real transfer
• Delta Action Model