SUFLECA: Scaling Up Feature Learning for CAD-to-image Alignment

2026-07-16

AI Summary

This work addresses the challenge of robustly estimating 9D object pose—including rotation, translation, and anisotropic scaling—from a single RGB image, particularly under occlusion and simulation-to-real domain shifts. The authors propose a weakly supervised, zero-shot CAD alignment framework that integrates pretrained vision foundation models, normalized object coordinate (NOC) supervision, and a geometrically consistent one-to-one matching mechanism to achieve efficient and accurate CAD-to-image alignment. Notably, this method is the first zero-shot approach to surpass fully supervised counterparts on ScanNet25k, achieving category- and instance-level accuracies of 33.4% and 42.3%, respectively—improving upon the strongest zero-shot baseline by 10.3 and 12.2 percentage points—and operates at sub-second inference speed per instance.

Abstract

CAD-to-image alignment aims to estimate an object's 9D pose (rotation, translation, and anisotropic scale) from a single RGB image, enabling applications in robotics and augmented reality. Recent zero-shot methods use visual foundation models to match image regions to CAD models, yet typically their correspondences are appearance-driven and degrade under occlusion or sim-to-real domain shift. To address these limitations, we introduce SUFLECA (Scaling Up Feature LEarning for CAD Alignment), a weakly-supervised framework for zero-shot CAD alignment with two key contributions. First, SUFLECA scales up geometry-grounded feature learning from pretrained visual representations through Normalized Object Coordinates (NOCs) supervision on 674K images spanning 12 real and synthetic datasets, learning compact geometry-aware features that generalize across domains. Second, we propose a geometrically consistent matching algorithm that establishes reliable one-to-one CAD-to-image correspondences. Together, these contributions enable accurate, sub-second alignment per object instance without iterative pose refinement. On ScanNet25k, SUFLECA achieves 33.4%/42.3% category/instance accuracy, outperforming, with a smaller computational footprint, the strongest zero-shot baseline by 10.3/12.2 percentage points and, for the first time on this benchmark, even surpassing fully supervised methods. Code is available at: https://github.com/snt-arg/SUFLECA

Problem

• CAD-to-image alignment
• 9D pose estimation
• zero-shot learning
• domain shift
• occlusion

Innovation

• CAD-to-image alignment
• zero-shot learning
• geometry-aware features
• Normalized Object Coordinates
• geometrically consistent matching