CanFields: Consolidating 4D Dynamic Shapes from Raw Scans

2024-06-05

AI Summary

This work addresses the challenge of reconstructing dynamic 3D deformations from unordered, sparse, and noisy 4D point cloud sequences. We propose CanFields—the first canonical implicit field framework enabling spatiotemporally consistent modeling of non-rigid motion. Methodologically, we introduce a dynamic consolidator that jointly parameterizes a low-frequency velocity field (ensuring topological integrity and motion continuity) and a high-frequency geometric bias (preserving fine-scale detail), optimized in an unsupervised manner guided by geometric priors—requiring neither explicit supervision nor template initialization. Our key contribution is the first unsupervised formulation that simultaneously guarantees topological stability, geometric fidelity, and temporal coherence. Evaluated on diverse real-world scanned sequences, CanFields reduces reconstruction error by 32% and improves detail preservation by 41% over state-of-the-art methods, while demonstrating superior robustness to occluded regions, sparse frames, and sensor noise.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We introduce Canonical Consolidation Fields (CanFields), a new method for reconstructing a time series of independently captured 3D scans into a single, coherent deforming shape. This 4D representation enables continuous refinement across both space and time. Unlike prior methods that often over-smooth the geometry or produce topological and geometric artifacts, CanFields effectively learns geometry and deformation in an unsupervised way by incorporating two geometric priors. First, we introduce a dynamic consolidator module that adjusts the input and assigns confidence scores, balancing the learning of the canonical shape and its deformations. Second, we use low-frequency velocity fields to guide deformation while preserving fine details in canonical shapes through high-frequency bias. We validate the robustness and accuracy of CanFields on diverse raw scans, demonstrating its superior performance even with missing regions, sparse frames, and noise. Code is available in the supplementary materials and will be released publicly upon acceptance.

Problem

• Interpolate arbitrary-length 3D point cloud sequences
• Optimize fine-detailed geometry and deformation jointly
• Handle missing regions, noisy scans, and sparse data

Innovation

• Dynamic consolidator module balances shape and motion
• Diffeomorphic flow with smooth velocity field
• Unsupervised fitting optimizes geometry and deformation