NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos

2025-11-11

AI Summary

This work addresses two key challenges in video-driven physics-based digital twin reconstruction of deformable objects: inaccurate physical modeling and poor generalization in future dynamic prediction. We propose a novel method integrating piecewise topological structure with a neural spring field. The former constructs a multi-region spring-connected topology via zeroth-order optimization, while the latter parameterizes spring constitutive relations using a canonical-coordinate neural network, explicitly encoding material heterogeneity and dynamic physical properties. Grounded in the mass-spring model, our approach synergistically combines zeroth-order optimization, insights from neural radiance fields, and coordinate-based neural networks to enhance spatially coherent physical modeling. Evaluated on real-world datasets, our method reduces Chamfer distance for current-state reconstruction by 20% and decreases physical prediction error for future frames by 25%, significantly outperforming state-of-the-art approaches.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this paper, we aim to create physical digital twins of deformable objects under interaction. Existing methods focus more on the physical learning of current state modeling, but generalize worse to future prediction. This is because existing methods ignore the intrinsic physical properties of deformable objects, resulting in the limited physical learning in the current state modeling. To address this, we present NeuSpring, a neural spring field for the reconstruction and simulation of deformable objects from videos. Built upon spring-mass models for realistic physical simulation, our method consists of two major innovations: 1) a piecewise topology solution that efficiently models multi-region spring connection topologies using zero-order optimization, which considers the material heterogeneity of real-world objects. 2) a neural spring field that represents spring physical properties across different frames using a canonical coordinate-based neural network, which effectively leverages the spatial associativity of springs for physical learning. Experiments on real-world datasets demonstrate that our NeuSping achieves superior reconstruction and simulation performance for current state modeling and future prediction, with Chamfer distance improved by 20% and 25%, respectively.

Problem

• Reconstructing deformable objects from video inputs
• Modeling intrinsic physical properties for realistic simulation
• Improving generalization from current state to future prediction

Innovation

• Neural spring field for deformable object reconstruction
• Piecewise topology modeling material heterogeneity
• Canonical coordinate network leveraging spatial associativity