Count Anything at Any Granularity

2026-05-11

AI Summary

This work addresses the challenge of open-world object counting, where ambiguous semantic granularity in user intent hinders the fulfillment of diverse counting requirements. To resolve this, the authors propose a multi-granularity counting framework that explicitly defines five levels of semantic granularity through visual exemplars and fine-grained textual prompts—including optional negative prompts—and introduces an automated data generation pipeline. Key contributions include the first explicit multi-granularity counting paradigm, the release of KubriCount—the first large-scale dataset enabling fine-grained semantic validation—and the design of HieraCount, a model that effectively fuses textual and visual exemplar guidance. Experiments demonstrate that HieraCount significantly improves counting accuracy and exhibits strong generalization in complex real-world scenarios, while also revealing a critical deficiency in existing models’ ability to adhere to fine-grained prompts.

Abstract

Open-world object counting remains brittle: despite rapid advances in vision-language models (VLMs), reliably counting the objects a user intends is far from solved. We argue that a central reason is that counting granularity is left implicit; users may refer to a specific identity, an attribute, an instance type, a category, or an abstract concept, yet most methods treat "what to count" as a single, category-level matching problem. In this work, we redefine open-world counting as multi-grained counting, where visual exemplars specify target appearance and fine-grained text, with optional negative prompts, specifies the intended semantic granularity across five explicit levels. Making granularity explicit, however, exposes a critical data bottleneck: existing counting datasets lack the multi-category scenes, controlled distractors, and instance-level annotations needed to verify fine-grained prompt semantics. To address this, we propose the first fully automatic data-scaling pipeline that integrates controllable 3D synthesis with consistent image editing and VLM-based filtering, and use it to construct KubriCount, the largest and most comprehensively annotated counting dataset to date, supporting both training and multi-grained evaluation. Systematic benchmarking reveals that both multimodal large language models and specialist counting models exhibit severe prompt-following failures under fine-grained distinctions. Motivated by these findings, we train HieraCount, a multi-grained counting model that jointly leverages text and visual exemplars as complementary target specifications. HieraCount substantially improves multi-grained counting accuracy and generalizes robustly to challenging real-world scenarios. The project page is available here: https://verg-avesta.github.io/KubriCount/.

Problem

• open-world object counting
• counting granularity
• fine-grained semantics
• visual exemplars
• prompt-following

Innovation

• multi-grained counting
• visual-language models
• automatic data synthesis
• KubriCount
• prompt-following