MooMIns -- Monocular 3D Reconstruction and Object Pose Estimation from Multiple Instances

2026-06-12

AI Summary

This work addresses the ill-posed problem of simultaneously reconstructing 3D geometry and estimating 6D poses for multiple object instances from a single monocular image. The authors propose a novel Gaussian splatting–based approach that leverages the implicit multi-view geometric information inherent in industrial scenes, where multiple instances of the same object category are randomly stacked. By inverting the conventional Gaussian splatting paradigm—rendering multiple instances from a single camera view rather than a single scene from multiple views—and integrating real geometric constraints, the method avoids hallucinations arising from data-driven depth priors. Combining SAM3 instance segmentation, an enhanced structure-from-motion pipeline, and Gaussian splatting–based geometric reconstruction, the approach achieves high-fidelity 3D reconstruction of unseen objects and robust 6D pose estimation for individual instances in both synthetic and real-world grasping scenarios.

Abstract

Simultaneous 3D reconstruction and 6D object pose estimation from a single monocular image is an inherently ill-posed problem. In industrial settings, however, multiple instances of an object are often randomly arranged in bins, implicitly providing several views of the same object within a single image. We show that this implicit multi-view geometry can be exploited to simultaneously reconstruct the object in 3D and estimate the 6D pose of each visible object instance. We present MooMIns, a new Gaussian-splatting-based approach that inverts the original Gaussian splatting formulation: instead of rendering a single scene from multiple cameras, we render multiple object instances from a single camera. Our method is initialized with SAM3 instance segmentation masks and a modified Structure from Motion (SfM) pipeline. In contrast to learned monocular depth estimation, we perform true geometry-based reconstruction from image evidence, avoiding hallucinations caused by training data priors. We evaluate MooMIns on synthetic and real bin-picking scenarios, and demonstrate accurate reconstruction of previously unseen objects as well as reliable pose estimation of individual instance

Problem

• monocular 3D reconstruction
• 6D object pose estimation
• multiple object instances
• ill-posed problem
• bin-picking

Innovation

• Gaussian splatting
• multi-instance 3D reconstruction
• 6D pose estimation
• monocular geometry
• Structure from Motion