GRACE: Gradient-Free Robot Action Generation via Combined Diffusion-MPPI Posterior Mean Estimation

2026-07-22

AI Summary

Existing diffusion-based policies struggle to handle non-differentiable constraints—such as collision avoidance and joint limits—during deployment, limiting their safe application in real-world robotics. This work proposes a novel approach that integrates a pretrained diffusion policy with an MPPI controller, which requires only forward cost evaluations. At each reverse diffusion step, the method constructs a cost-conditioned posterior and estimates its mean to guide action selection without relying on gradients. Notably, in differentiable settings, this framework naturally reduces to conventional gradient-based guidance, ensuring theoretical consistency. Experimental results demonstrate that the proposed method outperforms both diffusion and sampling-based baselines in simulation and successfully enables a real 7-DoF robotic arm to avoid obstacles, whereas unguided policies resulted in collisions in all trials.

Abstract

Diffusion policies generate multimodal robot action sequences from demonstrations, but steering them toward deployment-time constraints typically relies on differentiable guidance costs. This excludes many practical safety constraints, such as binary collision checks, joint limits, and black-box rollout costs that are nondifferentiable. We propose Gradient-free Robot Action generation via Combined diffusion-MPPI posterior mean Estimation (GRACE), which guides a pretrained diffusion policy with Model Predictive Path Integral (MPPI) control using only forward cost evaluations. Building on the common score-ascent structure of diffusion and MPPI, GRACE constructs a cost-conditioned guidance posterior at each reverse step and estimates its mean with a single MPPI update centered at the diffusion reverse mean. For differentiable costs, GRACE recovers conventional gradient guidance under a first-order, matched-covariance approximation. GRACE attains higher success rates than diffusion-based and sampling-based baselines in simulation. On a real 7-DoF manipulator, GRACE avoids a deployment-time obstacle that the unguided prior collides with in every trial. Code and experiment videos are available at https://anonymous.4open.science/w/grace-70BB/.

Problem

• diffusion policy
• non-differentiable constraints
• robot action generation
• safety constraints
• deployment-time guidance

Innovation

• diffusion policy
• MPPI
• gradient-free guidance
• posterior mean estimation
• nondifferentiable constraints