Corr2Distrib: Making Ambiguous Correspondences an Ally to Predict Reliable 6D Pose Distributions

2025-05-05

AI Summary

This work addresses the multimodal uncertainty inherent in RGB-based 6D pose estimation for symmetric and occluded objects. We propose the first method to model pose distributions via fuzzy 2D–3D correspondences. Our approach features three key contributions: (1) a symmetry-aware 3D point representation—comprising discriminative descriptors and local coordinate frames—that enables generating 3-DoF rotation hypotheses from individual 2D–3D matches; (2) explicit modeling of visual ambiguity as a multi-modal prior, rather than as isotropic noise; and (3) an end-to-end pipeline integrating weighted PnP optimization, confidence scoring, and distribution fitting to yield robust, multimodal pose distributions. Evaluated on the BOP benchmark, our method achieves state-of-the-art performance on both pose distribution estimation and deterministic pose estimation tasks, demonstrating superior robustness and generalization under symmetry and occlusion.

Technology Category

• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

We introduce Corr2Distrib, the first correspondence-based method which estimates a 6D camera pose distribution from an RGB image, explaining the observations. Indeed, symmetries and occlusions introduce visual ambiguities, leading to multiple valid poses. While a few recent methods tackle this problem, they do not rely on local correspondences which, according to the BOP Challenge, are currently the most effective way to estimate a single 6DoF pose solution. Using correspondences to estimate a pose distribution is not straightforward, since ambiguous correspondences induced by visual ambiguities drastically decrease the performance of PnP. With Corr2Distrib, we turn these ambiguities into an advantage to recover all valid poses. Corr2Distrib first learns a symmetry-aware representation for each 3D point on the object's surface, characterized by a descriptor and a local frame. This representation enables the generation of 3DoF rotation hypotheses from single 2D-3D correspondences. Next, we refine these hypotheses into a 6DoF pose distribution using PnP and pose scoring. Our experimental evaluations on complex non-synthetic scenes show that Corr2Distrib outperforms state-of-the-art solutions for both pose distribution estimation and single pose estimation from an RGB image, demonstrating the potential of correspondences-based approaches.

Problem

• Estimating 6D camera pose distributions from ambiguous RGB images
• Handling visual ambiguities like symmetries and occlusions in pose estimation
• Improving correspondence-based methods for reliable multi-pose solutions

Innovation

• Symmetry-aware 3D point representation learning
• Rotation hypotheses from single 2D-3D correspondences
• PnP refinement for 6DoF pose distribution