Test3R: Learning to Reconstruct 3D at Test Time

2025-06-16

AI Summary

Existing dense matching methods (e.g., DUSt3R) rely on pairwise predictions, leading to global geometric inconsistency and limited generalization. This paper introduces the first test-time self-optimization framework for multi-view 3D reconstruction: given image triplets as input, it reconstructs point clouds under a shared reference view via dual-pair reconstruction and employs differentiable cross-pair reprojection consistency as a self-supervised signal—enabling label-free, zero-annotation online fine-tuning. The method is plug-and-play, introducing only negligible parameters and computational overhead. It achieves state-of-the-art performance across both 3D reconstruction and multi-view depth estimation, significantly improving global structural accuracy. Moreover, it is model-agnostic—compatible with diverse backbone architectures—and requires only a few seconds per inference-time optimization.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Dense matching methods like DUSt3R regress pairwise pointmaps for 3D reconstruction. However, the reliance on pairwise prediction and the limited generalization capability inherently restrict the global geometric consistency. In this work, we introduce Test3R, a surprisingly simple test-time learning technique that significantly boosts geometric accuracy. Using image triplets ($I_1,I_2,I_3$), Test3R generates reconstructions from pairs ($I_1,I_2$) and ($I_1,I_3$). The core idea is to optimize the network at test time via a self-supervised objective: maximizing the geometric consistency between these two reconstructions relative to the common image $I_1$. This ensures the model produces cross-pair consistent outputs, regardless of the inputs. Extensive experiments demonstrate that our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multi-view depth estimation tasks. Moreover, it is universally applicable and nearly cost-free, making it easily applied to other models and implemented with minimal test-time training overhead and parameter footprint. Code is available at https://github.com/nopQAQ/Test3R.

Problem

• Improves 3D reconstruction geometric consistency
• Enhances cross-pair output consistency via self-supervised learning
• Reduces test-time training overhead and parameter footprint

Innovation

• Test-time learning for 3D reconstruction
• Self-supervised geometric consistency optimization
• Universal application with minimal overhead