BADGR: Bundle Adjustment Diffusion Conditioned by GRadients for Wide-Baseline Floor Plan Reconstruction

2025-03-25

AI Summary

Joint reconstruction of camera poses and floorplans from wide-baseline RGB panoramic images remains challenging due to sparse correspondences and geometric ambiguity. Method: We propose BADGR, an end-to-end diffusion model that uniquely conditions the denoising process on gradient signals derived from Levenberg–Marquardt optimization in bundle adjustment (BA), enabling joint refinement of pose graphs and wall layouts. BADGR integrates learnable structural priors—such as wall adjacency and collinearity—with multi-view geometric constraints, requiring only 2D floorplan supervision and supporting arbitrary input image density. Contribution/Results: Experiments demonstrate that BADGR significantly outperforms state-of-the-art methods across varying input densities, achieving substantial improvements in pose accuracy and floorplan structural fidelity. To our knowledge, this is the first work to validate the effectiveness of geometry-guided diffusion modeling—specifically, BA-derived gradients—as a conditioning signal for joint pose and floorplan reconstruction.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: State Estimation
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Reconstructing precise camera poses and floor plan layouts from wide-baseline RGB panoramas is a difficult and unsolved problem. We introduce BADGR, a novel diffusion model that jointly performs reconstruction and bundle adjustment (BA) to refine poses and layouts from a coarse state, using 1D floor boundary predictions from dozens of images of varying input densities. Unlike a guided diffusion model, BADGR is conditioned on dense per-entity outputs from a single-step Levenberg Marquardt (LM) optimizer and is trained to predict camera and wall positions while minimizing reprojection errors for view-consistency. The objective of layout generation from denoising diffusion process complements BA optimization by providing additional learned layout-structural constraints on top of the co-visible features across images. These constraints help BADGR to make plausible guesses on spatial relations which help constrain pose graph, such as wall adjacency, collinearity, and learn to mitigate errors from dense boundary observations with global contexts. BADGR trains exclusively on 2D floor plans, simplifying data acquisition, enabling robust augmentation, and supporting variety of input densities. Our experiments and analysis validate our method, which significantly outperforms the state-of-the-art pose and floor plan layout reconstruction with different input densities.

Problem

• Reconstructing camera poses and floor plans from wide-baseline panoramas
• Jointly refining poses and layouts using diffusion and bundle adjustment
• Mitigating errors in spatial relations with learned structural constraints

Innovation

• Diffusion model with gradient-conditioned bundle adjustment
• Levenberg Marquardt optimizer for dense per-entity outputs
• 2D floor plan training enabling robust augmentation