Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale

2025-10-02

AI Summary

Monocular non-rigid structure-from-motion (NRSfM) suffers from inaccurate depth estimation and inability to recover conformal scale, primarily due to restrictive assumptions—such as local planarity or linear deformation—that preclude conformal scale recovery. Method: We propose Con-NRSfM, a differential-geometry-based conformal NRSfM framework that enables decoupled and recoverable estimation of local conformal scale under conformal deformations. Our approach formulates a graph optimization problem integrating image-warping-guided pointwise 3D reconstruction with parallel separable iterative optimization, and incorporates a self-supervised encoder-decoder network to generate textured dense 3D point clouds. Contribution/Results: Con-NRSfM achieves state-of-the-art performance on both synthetic and real-world benchmarks, significantly improving reconstruction accuracy and robustness over existing methods. To our knowledge, it is the first method to enable explicit, decoupled conformal scale recovery in monocular NRSfM. The source code will be made publicly available.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Non-convex Optimization
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Non-rigid structure-from-motion (NRSfM), a promising technique for addressing the mapping challenges in monocular visual deformable simultaneous localization and mapping (SLAM), has attracted growing attention. We introduce a novel method, called Con-NRSfM, for NRSfM under conformal deformations, encompassing isometric deformations as a subset. Our approach performs point-wise reconstruction using 2D selected image warps optimized through a graph-based framework. Unlike existing methods that rely on strict assumptions, such as locally planar surfaces or locally linear deformations, and fail to recover the conformal scale, our method eliminates these constraints and accurately computes the local conformal scale. Additionally, our framework decouples constraints on depth and conformal scale, which are inseparable in other approaches, enabling more precise depth estimation. To address the sensitivity of the formulated problem, we employ a parallel separable iterative optimization strategy. Furthermore, a self-supervised learning framework, utilizing an encoder-decoder network, is incorporated to generate dense 3D point clouds with texture. Simulation and experimental results using both synthetic and real datasets demonstrate that our method surpasses existing approaches in terms of reconstruction accuracy and robustness. The code for the proposed method will be made publicly available on the project website: https://sites.google.com/view/con-nrsfm.

Problem

• Recovering conformal scale in non-rigid structure-from-motion
• Decoupling depth and conformal scale constraints for reconstruction
• Generating dense 3D point clouds from monocular deformable SLAM

Innovation

• Uses differential geometry for recoverable conformal scale
• Decouples depth and conformal scale constraints
• Employs self-supervised encoder-decoder for dense reconstruction