HccePose(BF): Predicting Front & Back Surfaces to Construct Ultra-Dense 2D-3D Correspondences for Pose Estimation

2025-10-11

AI Summary

Existing pose estimation methods predominantly focus on predicting 3D coordinates of the object’s front surface, neglecting the geometric modeling potential of the back surface and interior volume. This work proposes the Hierarchical Continuous Coordinate Encoding (HCCE), the first method to jointly model both front and back surfaces along with a dense volumetric representation between them. HCCE employs a neural network to predict front and back surface depths in tandem, followed by dense 3D sampling within the bounded volume to establish ultra-dense 2D–3D correspondences. The encoding enables high-precision, low-spectral-bias coordinate embedding, and is integrated with an optimized PnP solver to enhance pose robustness. Evaluated on all seven core datasets of the BOP benchmark, our approach achieves new state-of-the-art performance, improving average ADD(-S) accuracy by 2.1–5.7 percentage points—demonstrating the effectiveness and advancement of leveraging full 3D spatial geometry for pose estimation.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In pose estimation for seen objects, a prevalent pipeline involves using neural networks to predict dense 3D coordinates of the object surface on 2D images, which are then used to establish dense 2D-3D correspondences. However, current methods primarily focus on more efficient encoding techniques to improve the precision of predicted 3D coordinates on the object's front surface, overlooking the potential benefits of incorporating the back surface and interior of the object. To better utilize the full surface and interior of the object, this study predicts 3D coordinates of both the object's front and back surfaces and densely samples 3D coordinates between them. This process creates ultra-dense 2D-3D correspondences, effectively enhancing pose estimation accuracy based on the Perspective-n-Point (PnP) algorithm. Additionally, we propose Hierarchical Continuous Coordinate Encoding (HCCE) to provide a more accurate and efficient representation of front and back surface coordinates. Experimental results show that, compared to existing state-of-the-art (SOTA) methods on the BOP website, the proposed approach outperforms across seven classic BOP core datasets. Code is available at https://github.com/WangYuLin-SEU/HCCEPose.

Problem

• Predicting front and back object surfaces for pose estimation
• Creating ultra-dense 2D-3D correspondences using interior sampling
• Improving pose accuracy through hierarchical coordinate encoding

Innovation

• Predicts front and back surface 3D coordinates
• Creates ultra-dense 2D-3D correspondences via sampling
• Uses Hierarchical Continuous Coordinate Encoding technique