PhysMorph-GS: Differentiable Shape Morphing via Joint Optimization of Physics and Rendering Objectives

2025-11-21

AI Summary

Existing physics-based shape deformation methods suffer from a “rendering gap” because surface extraction is non-differentiable, preventing image-level losses from directly guiding physical optimization. This work proposes a physics-rendering jointly differentiable framework: it models physical dynamics via differentiable Material Point Method (MPM), introduces a differentiable particle-to-3D-Gaussian conversion bridge and deformation-aware upsampling to enable end-to-end differentiable mapping from physical states to multimodal renderings (RGB, depth, normals); and integrates multi-stage interleaved optimization with depth-enhanced supervision to close the rendering gap. Experiments demonstrate significant improvements in boundary fidelity and temporal stability. The method accurately reconstructs slender structures—such as ears and tails—in complex deformation sequences, achieving a ~2.5% reduction in Chamfer distance compared to pure physics-based baselines.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Shape morphing with physics-based simulation naturally supports large deformations and topology changes, but existing methods suffer from a "rendering gap": nondifferentiable surface extraction prevents image losses from directly guiding physics optimization. We introduce PhysMorph-GS, which couples a differentiable material point method (MPM) with 3D Gaussian splatting through a deformation-aware upsampling bridge that maps sparse particle states (x, F) to dense Gaussians (mu, Sigma). Multi-modal rendering losses on silhouette and depth backpropagate along two paths, from covariances to deformation gradients via a stretch-based mapping and from Gaussian means to particle positions. Through the MPM adjoint, these gradients update deformation controls while mass is conserved at a compact set of anchor particles. A multi-pass interleaved optimization scheme repeatedly injects rendering gradients into successive physics steps, avoiding collapse to purely physics-driven solutions. On challenging morphing sequences, PhysMorph-GS improves boundary fidelity and temporal stability over a differentiable MPM baseline and better reconstructs thin structures such as ears and tails. Quantitatively, our depth-supervised variant reduces Chamfer distance by about 2.5 percent relative to the physics-only baseline. By providing a differentiable particle-to-Gaussian bridge, PhysMorph-GS closes a key gap in physics-aware rendering pipelines and enables inverse design directly from image-space supervision.

Problem

• Bridging the rendering gap between physics simulation and image losses
• Enabling differentiable shape morphing with physics-based optimization
• Improving boundary fidelity and temporal stability in morphing sequences

Innovation

• Differentiable MPM coupled with Gaussian splatting
• Deformation-aware bridge maps particles to Gaussians
• Multi-pass optimization injects rendering into physics