ADEPT: Adaptive Diffusion Environment for Policy Transfer Sim-to-Real

2025-06-02

AI Summary

Existing outdoor environment generation methods rely heavily on heuristic parameters, limiting both diversity and realism. To address this, we propose an adaptive diffusion-based environment generation framework designed for zero-shot sim-to-real transfer. This work is the first to introduce denoising diffusion probabilistic models (DDPMs) into off-road navigation simulation environment generation. Our method employs policy-performance-weighted initial noise optimization and dynamic noise-level scheduling to enable fine-grained, policy-adaptive environmental evolution. Furthermore, it integrates multi-layer wilderness map representations with performance-weighted environmental mixing to dynamically expand the training scenario distribution. Experimental results demonstrate that navigation policies trained on our generated environments achieve significantly improved generalization and robustness compared to those trained on procedurally generated or real-terrain-simulated baselines—and outperform state-of-the-art navigation approaches.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Diffusion Models for Vision

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Model-free reinforcement learning has emerged as a powerful method for developing robust robot control policies capable of navigating through complex and unstructured environments. The effectiveness of these methods hinges on two essential elements: (1) the use of massively parallel physics simulations to expedite policy training, and (2) an environment generator tasked with crafting sufficiently challenging yet attainable environments to facilitate continuous policy improvement. Existing methods of outdoor environment generation often rely on heuristics constrained by a set of parameters, limiting the diversity and realism. In this work, we introduce ADEPT, a novel extbf{A}daptive extbf{D}iffusion extbf{E}nvironment for extbf{P}olicy extbf{T}ransfer in the zero-shot sim-to-real fashion that leverages Denoising Diffusion Probabilistic Models to dynamically expand existing training environments by adding more diverse and complex environments adaptive to the current policy. ADEPT guides the diffusion model's generation process through initial noise optimization, blending noise-corrupted environments from existing training environments weighted by the policy's performance in each corresponding environment. By manipulating the noise corruption level, ADEPT seamlessly transitions between generating similar environments for policy fine-tuning and novel ones to expand training diversity. To benchmark ADEPT in off-road navigation, we propose a fast and effective multi-layer map representation for wild environment generation. Our experiments show that the policy trained by ADEPT outperforms both procedural generated and natural environments, along with popular navigation methods.

Problem

• Enhancing sim-to-real policy transfer with adaptive diffusion environments
• Overcoming limited diversity in outdoor environment generation methods
• Improving robot navigation in complex unstructured off-road terrains

Innovation

• Uses Denoising Diffusion Probabilistic Models
• Optimizes initial noise for environment generation
• Adaptively blends noise-corrupted training environments