A Taxonomy of Structure from Motion Methods

2025-05-21

AI Summary

This paper addresses the Structure from Motion (SfM) problem—reconstructing 3D scene structure and camera motion from multi-view point correspondences. To overcome the lack of a unifying theoretical perspective in existing approaches, we propose the first taxonomy of SfM methods based on *problem emphasis*: whether the formulation prioritizes motion estimation, structure reconstruction, or their coupled optimization. Grounded in geometric computer vision and multi-view geometry, we formally model diverse problem settings and, for the first time, establish rigorous connections between SfM’s well-posedness conditions and its underlying modeling paradigms. Our framework enhances method interpretability and extensibility, clarifies fundamental theoretical bottlenecks and open challenges, and provides a unified conceptual foundation for algorithm design, theoretical analysis, and pedagogy.

Technology Category

• Computer Vision: Other Foundations of Computer Vision
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Structure from Motion (SfM) refers to the problem of recovering both structure (i.e., 3D coordinates of points in the scene) and motion (i.e., camera matrices) starting from point correspondences in multiple images. It has attracted significant attention over the years, counting practical reconstruction pipelines as well as theoretical results. This paper is conceived as a conceptual review of SfM methods, which are grouped into three main categories, according to which part of the problem - between motion and structure - they focus on. The proposed taxonomy brings a new perspective on existing SfM approaches as well as insights into open problems and possible future research directions. Particular emphasis is given on identifying the theoretical conditions that make SfM well posed, which depend on the problem formulation that is being considered.

Problem

• Recovering 3D scene structure and camera motion from images
• Classifying SfM methods based on motion or structure focus
• Identifying theoretical conditions for well-posed SfM problems

Innovation

• Recovering 3D structure and camera motion
• Grouping methods into three main categories
• Identifying theoretical conditions for SfM