🤖 AI Summary
This work addresses the challenges in industrial 6D object pose estimation—namely, its strong reliance on texture, requirement for extensive annotated data, and sensitivity to assembly defects—by introducing the first zero-shot framework capable of estimating the pose of unseen objects from RGB images using only textureless 3D models. The method leverages synthetically rendered depth and surface normal maps to align cross-modal features via pretraining, then combines 2D–3D keypoint back-projection, a correspondence filtering mechanism, and a PnP solver to robustly recover pose without object-specific training. Experiments demonstrate state-of-the-art performance on texture-deficient objects, and the authors release a new dataset featuring assembly defects, texture variations, and occlusions to validate real-world applicability.
📝 Abstract
6D pose estimation remains a key challenge in robotics and computer vision, particularly in industrial environments. The deployment of currently available data-driven methods is often limited by resource-intensive data pipelines, reliance on textured 3D models, and sensitivity to geometric deviations caused by damages or assembly defects. We present PIXIE, a zero-shot framework that estimates the 6D pose of an object from an RGB image using only an untextured 3D model. Synthetic depth and normal maps are rendered from sampled reference viewpoints and matched to the query image via a pretrained cross-modality feature matcher. Matched keypoints are back-projected to obtain 2D--3D correspondences for PnP-based pose estimation. Relying exclusively on geometry makes the method inherently robust to lighting and texture variation, while correspondence filtering handles geometric deviations between the model and physical object. We evaluate on widely-used public benchmarks, reporting state-of-the-art results on texture-less objects without object-specific training, and introduce a novel dataset with assembly defects, texture variations, and occlusion to demonstrate real-world applicability.