Dense-SfM: Structure from Motion with Dense Consistent Matching

2025-01-24

AI Summary

To address the low 3D reconstruction accuracy and trajectory discontinuity for textureless objects in consecutive image sequences, this paper proposes a novel method integrating dense matching with Gaussian Splatting (GS)-driven trajectory extension. We pioneer the fusion of dense correspondence estimation and GS-based long-range feature trajectory modeling, and design a multi-view kernelized matching module leveraging both Transformer architectures and Gaussian processes—significantly enhancing cross-view matching robustness and trajectory consistency. Furthermore, we incorporate multi-view geometric optimization to achieve end-to-end, high-density, high-accuracy 3D reconstruction. Evaluated on the ETH3D and Texture-Poor SfM benchmarks, our method surpasses state-of-the-art approaches in both point cloud density and reconstruction accuracy, demonstrating superior performance for texture-deficient scenes.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present Dense-SfM, a novel Structure from Motion (SfM) framework designed for dense and accurate 3D reconstruction from multi-view images. Sparse keypoint matching, which traditional SfM methods often rely on, limits both accuracy and point density, especially in texture-less areas. Dense-SfM addresses this limitation by integrating dense matching with a Gaussian Splatting (GS) based track extension which gives more consistent, longer feature tracks. To further improve reconstruction accuracy, Dense-SfM is equipped with a multi-view kernelized matching module leveraging transformer and Gaussian Process architectures, for robust track refinement across multi-views. Evaluations on the ETH3D and Texture-Poor SfM datasets show that Dense-SfM offers significant improvements in accuracy and density over state-of-the-art methods.

Problem

• 3D Reconstruction
• Moving Object
• Featureless Surfaces

Innovation

• Dense-SfM
• Multi-Angle Imaging
• Featureless Surface Reconstruction