Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning

2025-06-01

AI Summary

Diffusion models in long-horizon, sparse-reward offline reinforcement learning often deviate from the data manifold due to inaccurate guidance, generating infeasible trajectories—limiting their deployment in safety-critical applications. To address this, we propose LoMAP (Local Manifold Approximation and Projection), a training-free method that constructs a low-rank manifold subspace from offline data and applies real-time correction to diffusion-guided samples via local PCA and orthogonal projection, ensuring trajectory feasibility. We establish, for the first time, a theoretical lower bound linking the guidance gap to manifold deviation, thereby providing formal feasibility guarantees for diffusion-based planning. LoMAP is modular and seamlessly integrates with existing hierarchical diffusion planners. Empirical evaluation on standard offline RL benchmarks demonstrates significant improvements in both trajectory feasibility and task success rates.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Learning with Manifolds

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advances in diffusion-based generative modeling have demonstrated significant promise in tackling long-horizon, sparse-reward tasks by leveraging offline datasets. While these approaches have achieved promising results, their reliability remains inconsistent due to the inherent stochastic risk of producing infeasible trajectories, limiting their applicability in safety-critical applications. We identify that the primary cause of these failures is inaccurate guidance during the sampling procedure, and demonstrate the existence of manifold deviation by deriving a lower bound on the guidance gap. To address this challenge, we propose Local Manifold Approximation and Projection (LoMAP), a training-free method that projects the guided sample onto a low-rank subspace approximated from offline datasets, preventing infeasible trajectory generation. We validate our approach on standard offline reinforcement learning benchmarks that involve challenging long-horizon planning. Furthermore, we show that, as a standalone module, LoMAP can be incorporated into the hierarchical diffusion planner, providing further performance enhancements.

Problem

• Inconsistent reliability in diffusion-based generative planning
• Inaccurate guidance causing infeasible trajectory generation
• Manifold deviation during sampling procedure

Innovation

• Projects samples onto low-rank subspace
• Prevents infeasible trajectory generation
• Training-free manifold-aware diffusion planning