BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections

2026-07-04

AI Summary

This work addresses the challenge of reconstructing 3D structures of deformable objects from single images, which is hindered by the scarcity of large-scale datasets with paired 3D annotations. The authors propose a bootstrapping training framework that requires only a single skeletal canonical mesh per category and an unannotated collection of 2D images. Through iterative refinement of pointmap prediction, joint poses, and camera parameters—augmented by self-supervised synthetic data generated via rendering—the method progressively improves 3D shape estimation accuracy. By minimizing reliance on labor-intensive, fine-grained 3D annotations, this approach achieves reconstruction performance comparable to the strongly supervised DualPM method under weakly supervised conditions.

Abstract

3D reconstruction of articulated objects from a single image is challenging because large training datasets with paired image and 3D supervision are difficult to obtain. Recent point map-based methods achieve strong performance but rely on synthetic datasets rendered from manually created articulated 3D assets with carefully curated pose distributions. While camera viewpoints can be easily sampled, generating realistic object articulations remains costly and labor-intensive. We propose a training framework that reduces this requirement by leveraging unannotated 2D images collections with only a single rigged canonical mesh per category. Starting from a weak 3D shape predictor trained on canonical-pose renders, we iteratively estimate object articulation and camera pose by fitting the mesh to predicted point maps. The recovered articulations and viewpoints are then used to render updated synthetic training data, progressively improving the predictor. Despite using substantially weaker 3D supervision, our models achieve performance comparable with DualPM, which requires manually curated articulated training datasets.

Problem

• 3D reconstruction
• articulated objects
• single image
• weak supervision
• training data scarcity

Innovation

• bootstrapping
• articulated 3D reconstruction
• point maps
• weak supervision
• canonical mesh