MoAngelo: Motion-Aware Neural Surface Reconstruction for Dynamic Scenes

2025-09-19

AI Summary

High-fidelity neural surface reconstruction for dynamic scenes remains challenging due to low geometric fidelity, difficulties in modeling occlusions and topological changes. To address these issues, we propose a template-based dynamic reconstruction framework: first, NeuralAngelo is employed to generate an accurate initial geometry; then, a learnable non-rigid deformation field and a temporally consistent neural radiance field are jointly optimized. Crucially, we introduce a geometric refinement module that explicitly captures structural changes—such as limb separation and cloth wrinkles—that cannot be represented by the deformation field alone. Evaluated on the ActorsHQ dataset, our method significantly outperforms existing state-of-the-art approaches in both reconstruction accuracy and fine-detail preservation. It demonstrates superior robustness and geometric fidelity, particularly under complex articulation, severe occlusions, and topological evolution.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Motion & Tracking
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Dynamic scene reconstruction from multi-view videos remains a fundamental challenge in computer vision. While recent neural surface reconstruction methods have achieved remarkable results in static 3D reconstruction, extending these approaches with comparable quality for dynamic scenes introduces significant computational and representational challenges. Existing dynamic methods focus on novel-view synthesis, therefore, their extracted meshes tend to be noisy. Even approaches aiming for geometric fidelity often result in too smooth meshes due to the ill-posedness of the problem. We present a novel framework for highly detailed dynamic reconstruction that extends the static 3D reconstruction method NeuralAngelo to work in dynamic settings. To that end, we start with a high-quality template scene reconstruction from the initial frame using NeuralAngelo, and then jointly optimize deformation fields that track the template and refine it based on the temporal sequence. This flexible template allows updating the geometry to include changes that cannot be modeled with the deformation field, for instance occluded parts or the changes in the topology. We show superior reconstruction accuracy in comparison to previous state-of-the-art methods on the ActorsHQ dataset.

Problem

• Reconstructing dynamic scenes from multi-view videos
• Overcoming noisy and smooth mesh extraction issues
• Extending static neural surface methods to dynamic settings

Innovation

• Extends NeuralAngelo to dynamic scenes
• Jointly optimizes deformation fields and template
• Flexible template updates geometry for changes