Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models

2026-08-11

AI Summary

This work addresses the inconsistency in depth and camera pose estimation arising from the absence of explicit multi-view geometric constraints in existing foundation vision models. To overcome this limitation without requiring ground-truth 3D labels, the authors propose a plug-and-play test-time adaptation method that leverages 2D pixel correspondences as pseudo-supervision to explicitly enforce epipolar geometry constraints and multi-view consistency losses. The approach innovatively integrates SO(3) geodesic distance–guided view sampling, gradient decoupling for optimization, and a lightweight LoRA fine-tuning mechanism to jointly refine geometry and pose estimates. Extensive experiments demonstrate consistent and significant improvements across six prominent vision foundation models and four benchmark datasets, underscoring the method’s generality and effectiveness.

Abstract

Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization, achieving strong generalization. However, enforcing explicit multi-view geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining, so such inconsistency can arise. To address this, implicit self-consistency derived from model outputs (e.g., pointmaps, features), though enforced at test-time in prior work, delivers inherently limited performance gain, especially on scenes where the pretrained VFM is highly inaccurate. In contrast to this implicit signal, we propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth. Our proposed Self-Geometry consists of Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with Gradient Disentanglement to prevent gradient conflict; Frame Angular-Neighbor, a view sampler based on SO(3) geodesic distances for lightly imposing these constraints; and Lightweight TTA, which adapts VFMs via LoRA. Our method achieves consistent improvements in both pose and geometry estimation across six VFMs (VGGT, $π^3$, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).

Problem

• geometric consistency
• vision foundation models
• test-time adaptation
• multi-view constraints
• 3D vision

Innovation

• Self-Geometry
• Test-Time Adaptation
• Geometric Consistency
• Vision Foundation Models
• LoRA