Counting Stacked Objects from Multi-View Images

2024-11-28

arXiv.org

AI Summary

Counting densely stacked, heavily occluded 3D objects—such as identical containers inside a cargo hold—remains a challenging problem due to severe inter-object occlusion and depth ambiguity. To address this, we propose the first end-to-end trainable multi-view 3D counting framework. Our core innovation lies in decoupling counting into two complementary subtasks: geometric reconstruction and voxel occupancy ratio estimation, jointly optimized under multi-view geometric consistency constraints, depth map learning, and 3D voxelized scene modeling. Unlike conventional 2D or single-view approaches, our method explicitly models volumetric occupancy across views, enabling robust counting under extreme occlusion. Extensive experiments on both real-world scenes and a large-scale synthetic dataset demonstrate state-of-the-art performance, reducing average counting error by 42% over prior methods. To foster reproducibility and further research, we publicly release all datasets and source code.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Visual object counting is a fundamental computer vision task underpinning numerous real-world applications, from cell counting in biomedicine to traffic and wildlife monitoring. However, existing methods struggle to handle the challenge of stacked 3D objects in which most objects are hidden by those above them. To address this important yet underexplored problem, we propose a novel 3D counting approach that decomposes the task into two complementary subproblems - estimating the 3D geometry of the object stack and the occupancy ratio from multi-view images. By combining geometric reconstruction and deep learning-based depth analysis, our method can accurately count identical objects within containers, even when they are irregularly stacked. We validate our 3D Counting pipeline on diverse real-world and large-scale synthetic datasets, which we will release publicly to facilitate further research.

Problem

• Accurately count stacked 3D objects
• Estimate 3D geometry and occupancy ratio
• Handle hidden objects in irregular stacks

Innovation

• Decomposes counting into geometry and occupancy estimation
• Combines geometric reconstruction with deep learning
• Validated on real-world and synthetic datasets