Robust 3D Alignment of Generative Reconstructions via Partial Monocular Observations

2026-07-01

AI Summary

This work addresses the challenging alignment problem between generative 3D reconstructions and sparse, noisy monocular observations, which is hindered by scale ambiguity, geometric hallucinations, and initial lack of overlap. The authors propose a training-free geometric alignment framework that recovers metric scale and pose via Sim(3) transformation through a coarse-to-fine strategy for robust initialization and precise refinement. Key innovations include an explicit scale factor to resolve scale ambiguity, a geometry-aware descriptor paired with a decoupled closed-form solver, and a hallucination filtering mechanism to suppress spurious geometry generated by neural models. Evaluated on the newly introduced GenPMOAlign–Where2Place benchmark, the method significantly outperforms both classical geometric and state-of-the-art learning-based approaches, achieving stable and highly accurate alignment.

Abstract

Aligning generative 3D reconstructions with partial monocular observations is a critical but under-explored challenge in computer vision. This task is inherently ill-posed due to severe asymmetries between noisy, sparse monocular inputs and dense generative priors, whose scale ambiguity and geometric hallucinations, combined with the lack of initial overlap, render traditional registration pipelines ineffective. To resolve these issues, we propose a training-free and interpretable geometric alignment framework that grounds generative 3D priors via a 3D similarity transformation (Sim(3)), which can recover accurate metric scale and pose. Specifically, we introduce an explicit scale factor to resolve metric ambiguity and employ a coarse-to-fine alignment strategy, leveraging geometry-aware descriptors for robust initialization and a decoupled closed-form solver for precision refinement. In addition, we introduce a Hallucination Filtering operation to effectively suppress outliers caused by hallucinated geometry. To evaluate alignment performance under these extreme conditions, we introduce GenPMOAlign--Where2Place, a rigorous benchmark specifically designed for Generative-to-Partial Monocular Observational Alignment. Experiments demonstrate that our method achieves stable and accurate registration, substantially outperforming both classical geometric pipelines and state-of-the-art learning-based baselines. Code and the benchmark will be publicly released.

Problem

• 3D alignment
• generative reconstruction
• monocular observation
• scale ambiguity
• geometric hallucination

Innovation

• generative 3D reconstruction
• monocular alignment
• Sim(3) registration
• hallucination filtering
• coarse-to-fine alignment