Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects

2026-05-27

AI Summary

This work addresses the challenge of 9D pose estimation for everyday objects in single real-world images, which has been hindered by the scarcity of large-scale, accurately annotated real data. The authors construct a large-scale real-world 9D pose dataset comprising 21.8 million images across 700 object categories—the first to achieve annotation at the tens-of-millions scale. They reconstruct point clouds via multi-view geometry and propagate canonical poses from a small set of human-annotated instances through cross-instance alignment. Additionally, they model category-level symmetries by introducing cross-category orientation priors. The resulting dataset exceeds the largest existing real 9D pose dataset by two orders of magnitude and significantly improves model generalization on benchmarks including ImageNet3D, PASCAL3D+, and HANDAL, outperforming current state-of-the-art methods.

Abstract

Estimating the 9D pose of everyday objects from a single real-world image remains challenging. This is largely due to the lack of large-scale supervision. Most existing datasets either rely heavily on synthetic renderings or provide limited coverage of real-world objects: the largest real-world 9D pose dataset to date contains only 17K annotated objects across 9 categories. We address this gap with Every9D-21M, a dataset of 9D pose annotations for 21.8M real-world images from 109K object- centric videos spanning 700 everyday object categories - two orders of magnitude larger than prior real-world 9D pose benchmarks in both image and category count. To achieve this scale, we leverage object-centric videos by reconstructing object- level point clouds via multi-view geometry and aligning similar instances into a shared canonical coordinate frame. Canonical poses are manually annotated for only a small set of reference objects (fewer than 0.01% of all images) and propagated to the remaining instances via cross-instance alignment. All propagated canonical poses are then verified from multiple viewpoints. We further introduce cross-category orientation rules that induce category-level symmetries, enabling symmetry-aware evaluation. Beyond establishing dedicated training and evaluation splits as a benchmark for 9D pose foundation models, we show that training on Every9D-21M improves performance on ImageNet3D and PASCAL3D+, and generalizes to HANDAL substantially better than training on ImageNet3D. Data and code are available at https://github.com/GenIntel/Every9D.

Problem

• 9D pose estimation
• real-world objects
• large-scale dataset
• canonicalization
• object pose supervision

Innovation

• 9D pose canonicalization
• large-scale real-world dataset
• cross-instance alignment
• category-level symmetry
• object-centric video reconstruction