🤖 AI Summary
Non-rigid structure-from-motion (NRSfM) suffers from two fundamental challenges: the inherent ambiguity between camera motion and non-rigid deformation, and the inadequacy of global low-rank assumptions in modeling intense, anisotropic local deformations. To address these, we propose a spatiotemporal co-modeling framework: (1) a novel reference-free temporal smoothing Procrustean alignment module that explicitly decouples camera motion from shape deformation; and (2) a spatially adaptive weighted low-rank constraint enabling robust modeling of heterogeneous, large-magnitude deformations. Our method eliminates the need for a predefined reference configuration, thereby avoiding error accumulation inherent in conventional alignment strategies. Evaluated on multiple standard benchmarks, our approach consistently outperforms existing low-rank NRSfM methods—achieving superior deformation accuracy, enhanced robustness in motion-deformation disentanglement, and improved capability in reconstructing complex non-isometric motions.
📝 Abstract
Even though Non-rigid Structure-from-Motion (NRSfM) has been extensively studied and great progress has been made, there are still key challenges that hinder their broad real-world applications: 1) the inherent motion/rotation ambiguity requires either explicit camera motion recovery with extra constraint or complex Procrustean Alignment; 2) existing low-rank modeling of the global shape can over-penalize drastic deformations in the 3D shape sequence. This paper proposes to resolve the above issues from a spatial-temporal modeling perspective. First, we propose a novel Temporally-smooth Procrustean Alignment module that estimates 3D deforming shapes and adjusts the camera motion by aligning the 3D shape sequence consecutively. Our new alignment module remedies the requirement of complex reference 3D shape during alignment, which is more conductive to non-isotropic deformation modeling. Second, we propose a spatial-weighted approach to enforce the low-rank constraint adaptively at different locations to accommodate drastic spatially-variant deformation reconstruction better. Our modeling outperform existing low-rank based methods, and extensive experiments across different datasets validate the effectiveness of our method1. 1 Project page: https://npucvr.github.io/TSM-NRSfM.