SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space

2025-07-02

AI Summary

Existing diffusion policies exhibit limited generalization to novel 3D object arrangements, hindering real-world deployment. To address this, we propose the Spherical Diffusion Policy (SDP), the first diffusion-based policy modeled in spherical Fourier space to enforce strict SE(3) equivariance. SDP achieves this via a spherical FiLM conditioning mechanism and a spatiotemporally equivariant spherical U-Net architecture, enabling adaptive policy behavior under arbitrary rigid-body transformations of 3D scenes. Our approach integrates spherical Fourier transforms, SE(3)-equivariant neural networks, and a diffusion-based denoising process to learn closed-loop manipulation policies end-to-end. Evaluated on 20 simulated and 5 real-robot tasks—including both single-arm and bimanual operations—SDP consistently outperforms state-of-the-art baselines. Results demonstrate superior generalization to unseen spatial configurations and validate feasibility for physical deployment.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Diffusion Policies are effective at learning closed-loop manipulation policies from human demonstrations but generalize poorly to novel arrangements of objects in 3D space, hurting real-world performance. To address this issue, we propose Spherical Diffusion Policy (SDP), an SE(3) equivariant diffusion policy that adapts trajectories according to 3D transformations of the scene. Such equivariance is achieved by embedding the states, actions, and the denoising process in spherical Fourier space. Additionally, we employ novel spherical FiLM layers to condition the action denoising process equivariantly on the scene embeddings. Lastly, we propose a spherical denoising temporal U-net that achieves spatiotemporal equivariance with computational efficiency. In the end, SDP is end-to-end SE(3) equivariant, allowing robust generalization across transformed 3D scenes. SDP demonstrates a large performance improvement over strong baselines in 20 simulation tasks and 5 physical robot tasks including single-arm and bi-manual embodiments. Code is available at https://github.com/amazon-science/Spherical_Diffusion_Policy.

Problem

• Improves generalization of diffusion policies in 3D space
• Achieves SE(3) equivariance via spherical Fourier space
• Enhances robustness in transformed 3D scenes

Innovation

• SE(3)-equivariant diffusion policy in Fourier space
• Spherical FiLM layers for equivariant conditioning
• Spherical denoising U-net for spatiotemporal equivariance