Joint Model-based Model-free Diffusion for Planning with Constraints

2025-09-10

AI Summary

Poor compatibility and difficulty optimizing multimodal outputs arise when integrating model-free diffusion-based planning with model-based safety-constrained modules. Method: We propose JM2D, a joint sampling framework that unifies both components into a single differentiable sampling problem. JM2D introduces a differentiable interaction potential function for end-to-end guidance and incorporates importance sampling to optimize non-differentiable, non-convex objectives—without requiring additional training. Theoretically, conditional diffusion generation is shown to be a special case of JM2D. Results: Evaluated on offline reinforcement learning and robotic manipulation tasks, JM2D significantly improves task success rates while strictly enforcing safety constraints. It outperforms conventional safety-filtering approaches in both efficacy and constraint satisfaction, demonstrating superior robustness and generalization across diverse control scenarios.

Technology Category

• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Model-free diffusion planners have shown great promise for robot motion planning, but practical robotic systems often require combining them with model-based optimization modules to enforce constraints, such as safety. Naively integrating these modules presents compatibility challenges when diffusion's multi-modal outputs behave adversarially to optimization-based modules. To address this, we introduce Joint Model-based Model-free Diffusion (JM2D), a novel generative modeling framework. JM2D formulates module integration as a joint sampling problem to maximize compatibility via an interaction potential, without additional training. Using importance sampling, JM2D guides modules outputs based only on evaluations of the interaction potential, thus handling non-differentiable objectives commonly arising from non-convex optimization modules. We evaluate JM2D via application to aligning diffusion planners with safety modules on offline RL and robot manipulation. JM2D significantly improves task performance compared to conventional safety filters without sacrificing safety. Further, we show that conditional generation is a special case of JM2D and elucidate key design choices by comparing with SOTA gradient-based and projection-based diffusion planners. More details at: https://jm2d-corl25.github.io/.

Problem

• Integrating model-free diffusion with model-based optimization for constraints
• Addressing compatibility challenges between multi-modal outputs and optimization modules
• Handling non-differentiable objectives from non-convex optimization modules

Innovation

• Joint sampling framework for module compatibility
• Importance sampling guides non-differentiable optimization modules
• Safety-aligned diffusion planning without additional training