SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation
🤖 AI Summary
Existing simulators struggle to accurately model the dynamics of deformable objects driven jointly by environmental interactions and robotic actions, and they often lack effective support for action conditioning, limiting their accuracy, stability, and generalization. This work proposes SoMA—the first neural simulator based on 3D Gaussian splatting—that unifies the modeling of deformable object dynamics, environmental forces, and robot joint actions within an implicit neural space. For the first time, robotic action conditions are embedded directly into the dynamic Gaussian splatting framework, enabling end-to-end simulation of real-to-sim soft manipulation without requiring predefined physical rules. Experiments demonstrate that SoMA improves re-simulation accuracy and generalization by 20% on real-world soft manipulation tasks and successfully achieves stable long-horizon simulations of complex operations such as cloth folding.
Application Category
📝 Abstract
Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding.
Problem
Research questions and friction points this paper is trying to address.
soft-body manipulation
deformable object simulation
real-to-sim
robotic interaction
dynamics modeling
Innovation
Methods, ideas, or system contributions that make the work stand out.
real-to-sim
soft-body manipulation
3D Gaussian Splatting
neural simulation
deformable dynamics
🔎 Similar Papers
No similar papers found.
