PicoPose: Progressive Pixel-to-Pixel Correspondence Learning for Novel Object Pose Estimation

2025-04-03

AI Summary

This paper addresses zero-shot 6D pose estimation: accurately estimating the 6D pose of CAD objects unseen during training, using only RGB images. We propose a three-stage progressive pixel-wise correspondence learning framework—comprising template matching, global affine transformation regression, and local offset refinement—which decouples and jointly optimizes geometric modeling across stages to significantly enhance cross-object generalization. Our method integrates differentiable template rendering, CNN-based feature matching, 2D affine parameter regression, and window-based offset prediction, followed by PnP/RANSAC for pose solving. Evaluated on all seven BOP benchmark datasets, it achieves state-of-the-art performance. Crucially, it demonstrates strong zero-shot generalization to both unseen CAD models and novel reference images, establishing a new paradigm for open-set 6D pose estimation.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Novel object pose estimation from RGB images presents a significant challenge for zero-shot generalization, as it involves estimating the relative 6D transformation between an RGB observation and a CAD model of an object that was not seen during training. In this paper, we introduce PicoPose, a novel framework designed to tackle this task using a three-stage pixel-to-pixel correspondence learning process. Firstly, PicoPose matches features from the RGB observation with those from rendered object templates, identifying the best-matched template and establishing coarse correspondences. Secondly, PicoPose smooths the correspondences by globally regressing a 2D affine transformation, including in-plane rotation, scale, and 2D translation, from the coarse correspondence map. Thirdly, PicoPose applies the affine transformation to the feature map of the best-matched template and learns correspondence offsets within local regions to achieve fine-grained correspondences. By progressively refining the correspondences, PicoPose significantly improves the accuracy of object poses computed via PnP/RANSAC. PicoPose achieves state-of-the-art performance on the seven core datasets of the BOP benchmark, demonstrating exceptional generalization to novel objects represented by CAD models or object reference images. Code and models are available at https://github.com/foollh/PicoPose.

Problem

• Estimating 6D pose of unseen objects from RGB images
• Establishing pixel-to-pixel correspondences for novel objects
• Improving pose accuracy via progressive correspondence refinement

Innovation

• Three-stage pixel-to-pixel correspondence learning
• Global regression of 2D affine transformation
• Local fine-grained correspondence offset learning