DP-SfM: Dual-Pixel Structure-from-Motion without Scale Ambiguity

2026-05-03

AI Summary

This work addresses the scale ambiguity inherent in multi-view 3D reconstruction, which typically requires known reference objects to recover absolute scale. The authors propose a novel method that eliminates the need for reference objects or prior calibration by leveraging defocus blur cues from dual-pixel (DP) sensor images together with scale-ambiguous depth maps. Through linear scale estimation and blur-kernel-based cross-view intensity optimization, the approach automatically resolves absolute scale directly from DP imagery. To the best of our knowledge, this is the first technique capable of disambiguating scale in structure-from-motion (SfM) using only DP images, enabling high-accuracy metric-scale reconstruction. Extensive experiments across diverse cameras, lenses, and scenes demonstrate the method’s effectiveness and robustness.

Abstract

Multi-view 3D reconstruction, namely, structure-from-motion followed by multi-view stereo, is a fundamental component of 3D computer vision. In general, multi-view 3D reconstruction suffers from an unknown scale ambiguity unless a reference object of known size is present in the scene. In this article, we show that multi-view images captured using a dual-pixel (DP) sensor can automatically resolve the scale ambiguity, without requiring a reference object or prior calibration. Specifically, the defocus blur observed in DP images provides sufficient information to determine the absolute scale when paired with depth maps (up to scale) recovered from multi-view 3D reconstruction. Based on this observation, we develop a simple yet effective linear method to estimate the absolute scale, followed by the intensity-based optimization stage that aligns the left and right DP images by shifting them back toward each other using cross-view blur kernels. Experiments demonstrate the effectiveness of the proposed approach across diverse scenes captured with different cameras and lenses. Code and data are available at https://github.com/lilika-makabe/dp-sfm-tpami.git

Problem

• scale ambiguity
• structure-from-motion
• multi-view 3D reconstruction
• dual-pixel sensor

Innovation

• Dual-Pixel
• Structure-from-Motion
• Scale Ambiguity
• Defocus Blur
• Multi-view 3D Reconstruction