Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting

2026-02-06

AI Summary

This work addresses the poor generalization of monocular vision–based drone navigation in unstructured outdoor environments, primarily caused by illumination discrepancies between simulation and reality. To bridge this gap, the authors construct a high-fidelity simulation environment grounded in real-world data and propose a relightable 3D Gaussian splatting method that decouples geometry from lighting, enabling physically plausible illumination editing. Integrated with end-to-end reinforcement learning, the framework directly maps monocular RGB images to continuous control commands. The resulting system supports zero-shot transfer, allowing a lightweight quadrotor to achieve robust, collision-free navigation in real forest environments at speeds up to 10 m/s without any fine-tuning, significantly enhancing adaptability to dynamic lighting variations.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

UAV navigation in unstructured outdoor environments using passive monocular vision is hindered by the substantial visual domain gap between simulation and reality. While 3D Gaussian Splatting enables photorealistic scene reconstruction from real-world data, existing methods inherently couple static lighting with geometry, severely limiting policy generalization to dynamic real-world illumination. In this paper, we propose a novel end-to-end reinforcement learning framework designed for effective zero-shot transfer to unstructured outdoors. Within a high-fidelity simulation grounded in real-world data, our policy is trained to map raw monocular RGB observations directly to continuous control commands. To overcome photometric limitations, we introduce Relightable 3D Gaussian Splatting, which decomposes scene components to enable explicit, physically grounded editing of environmental lighting within the neural representation. By augmenting training with diverse synthesized lighting conditions ranging from strong directional sunlight to diffuse overcast skies, we compel the policy to learn robust, illumination-invariant visual features. Extensive real-world experiments demonstrate that a lightweight quadrotor achieves robust, collision-free navigation in complex forest environments at speeds up to 10 m/s, exhibiting significant resilience to drastic lighting variations without fine-tuning.

Problem

• UAV navigation
• visual domain gap
• illumination variation
• zero-shot transfer
• unstructured environments

Innovation

• Relightable 3D Gaussian Splatting
• zero-shot navigation
• illumination-invariant policy
• monocular vision
• reinforcement learning