Long-tail Internet photo reconstruction

2026-04-24

AI Summary

This work addresses the challenge of 3D reconstruction from internet photos in long-tailed scenarios, where sparse image coverage, uneven viewpoints, and noise severely degrade performance. The authors propose a novel approach based on fine-tuning 3D foundation models, introducing a sparse camera sampling strategy derived from high-quality reconstructions of well-known landmarks. They construct a new dataset, MegaDepth-X, which for the first time effectively simulates realistic supervision signals for long-tailed scenes. The method substantially improves reconstruction robustness under extreme sparsity, symmetry, and repetitive structures, while maintaining strong generalization on standard dense 3D benchmarks, thereby surpassing the performance limitations of existing approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Internet photo collections exhibit an extremely long-tailed distribution: a few famous landmarks are densely photographed and easily reconstructed in 3D, while most real-world sites are represented with sparse, noisy, uneven imagery beyond the capabilities of both classical and learned 3D methods. We believe that tackling this long-tail regime represents one of the next frontiers for 3D foundation models. Although reliable ground-truth 3D supervision from sparse scenes is challenging to acquire, we observe that it can be effectively simulated by sampling sparse subsets from well-reconstructed Internet landmarks. To this end, we introduce MegaDepth-X, a large dataset of 3D reconstructions with clean, dense depth, together with a strategy for sampling sets of training images that mimic camera distributions in long-tail scenes. Finetuning 3D foundation models with these components yields robust reconstructions under extreme sparsity, and also enables more reliable reconstruction in symmetric and repetitive scenes, while preserving generalization to standard, dense 3D benchmark datasets.

Problem

• long-tail
• 3D reconstruction
• sparse imagery
• Internet photo collections
• 3D foundation models

Innovation

• long-tail 3D reconstruction
• sparse image sampling
• 3D foundation models
• MegaDepth-X
• camera distribution simulation