Counting Through Occlusion: Framework for Open World Amodal Counting

2025-11-16

AI Summary

Existing object counting methods suffer significant performance degradation under occlusion due to backbone networks encoding occluding surfaces rather than true objects, leading to critical feature distortion. To address this, we propose CountOCC—the first amodal framework explicitly designed for open-world occlusion-robust counting. Its core contributions are: (1) a hierarchical multimodal guidance mechanism that fuses textual semantic priors with visual embeddings; (2) pyramid-based feature reconstruction coupled with spatial context fusion to explicitly recover object representations in occluded regions; and (3) visual equivalence constraints and attention-based spatial consistency optimization to preserve geometric fidelity of features before and after occlusion. Evaluated on FSC147, CARPK, and CAPTUREReal, CountOCC reduces mean absolute error (MAE) by 26.72%, 49.89%, and 28.79%, respectively, achieving new state-of-the-art performance across all benchmarks.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Object counting has achieved remarkable success on visible instances, yet state-of-the-art (SOTA) methods fail under occlusion, a pervasive challenge in real world deployment. This failure stems from a fundamental architectural limitation where backbone networks encode occluding surfaces rather than target objects, thereby corrupting the feature representations required for accurate enumeration. To address this, we present CountOCC, an amodal counting framework that explicitly reconstructs occluded object features through hierarchical multimodal guidance. Rather than accepting degraded encodings, we synthesize complete representations by integrating spatial context from visible fragments with semantic priors from text and visual embeddings, generating class-discriminative features at occluded locations across multiple pyramid levels. We further introduce a visual equivalence objective that enforces consistency in attention space, ensuring that both occluded and unoccluded views of the same scene produce spatially aligned gradient-based attention maps. Together, these complementary mechanisms preserve discriminative properties essential for accurate counting under occlusion. For rigorous evaluation, we establish occlusion-augmented versions of FSC 147 and CARPK spanning both structured and unstructured scenes. CountOCC achieves SOTA performance on FSC 147 with 26.72% and 20.80% MAE reduction over prior baselines under occlusion in validation and test, respectively. CountOCC also demonstrates exceptional generalization by setting new SOTA results on CARPK with 49.89% MAE reduction and on CAPTUREReal with 28.79% MAE reduction, validating robust amodal counting across diverse visual domains. Code will be released soon.

Problem

• Addressing object counting failures under occlusion conditions
• Reconstructing occluded object features using multimodal guidance
• Improving counting accuracy across diverse visual domains

Innovation

• Reconstructs occluded features via multimodal guidance
• Enforces visual consistency through attention space alignment
• Generates class-discriminative features across pyramid levels