MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos

2026-05-28

AI Summary

Monocular videos lack multi-view geometric constraints, making it challenging to accurately recover the geometry, appearance, and physical parameters of deformable objects. This work proposes a joint optimization framework that integrates differentiable Material Point Method (MPM) physics simulation with 3D Gaussian Splatting. Through three key innovations—global scale alignment, physics-aware geometric optimization, and a differentiable position map—the method achieves, for the first time, high-fidelity inverse physics estimation from monocular input alone. Evaluated on the Vid2Sim benchmark and a newly introduced dataset of elastoplastic objects, the approach significantly outperforms existing monocular methods and attains performance comparable to multi-view baselines.

Abstract

Existing inverse physics methods recover physical parameters from multi-view videos, where geometric constraints across views resolve scale and 3D structure. In monocular settings, however, such constraints are absent, leading to severe scale ambiguity, inaccurate geometry, and weak coupling between appearance optimization and physical simulation. We propose MonoPhysics, a framework for monocular inverse physics estimation of deformable objects using differentiable MPM simulation and 3D Gaussian Splatting, which jointly optimizes geometry, appearance, and physical parameters from a single camera view. We address these challenges through three visual-physical bridges: global scale alignment, physics-aware geometry refinement, and a differentiable position map, which together enable accurate optimization from monocular observations alone. We evaluate on Vid2Sim and our new dataset of elastic and plastic objects, showing that MonoPhysics outperforms existing baselines in monocular settings and achieves performance comparable to multi-view baselines using only a single camera. Our project page is available at https://daniel03c1.github.io/MonoPhysics/

Problem

• monocular video
• scale ambiguity
• geometry estimation
• physical parameter estimation
• appearance optimization

Innovation

• monocular inverse physics
• differentiable MPM simulation
• 3D Gaussian Splatting
• scale ambiguity resolution
• physics-aware geometry refinement