Test-Time Training for Robust Text-Guided Open-Vocabulary Object Counting

2026-06-16

AI Summary

Existing text-guided open-vocabulary object counting methods perform well under ideal conditions but exhibit significantly degraded robustness in real-world image degradation scenarios such as rain, fog, low light, and noise. To address this limitation, this work proposes the Dual-TTT framework, which, at test time, trains only a lightweight text-guided denoising module (TL-Denoiser) while keeping the original counting network frozen. This approach enables end-to-end optimization without requiring additional annotations or architectural modifications. The method introduces Robust-TOOC, the first benchmark specifically designed for evaluating robustness in text-guided open-vocabulary object counting, and demonstrates substantial improvements in counting accuracy and robustness across six types of image degradations for multiple state-of-the-art TOOC models.

Abstract

Text-guided Open-vocabulary Object Counting (TOOC) enables counting arbitrary object categories specified by text prompts, offering substantially greater flexibility than conventional closed-set counting. However, existing TOOC methods are developed and evaluated primarily on ideal images, while real-world scenes often suffer from adverse conditions such as rain, fog, darkness, and sensor noise, which severely degrade visual quality and impair vision-language alignment. To bridge this gap, we introduce Robust-TOOC, the first benchmark for evaluating TOOC under diverse corruption conditions, which covers six representative degradation types: rain, fog, darkness, Gaussian noise, salt-and-pepper noise, and mixed corruption. To improve robustness while preserving the original counting architecture, we propose Dual-TTT, a dual-architecture test-time training framework for TOOC. Specifically, during test-time training, Dual-TTT updates only the Text-guided Lightweight Denoising module (TL-Denoiser), while keeping the original counting network frozen. Inspired by diffusion models, the TL-Denoiser is optimized to remove corruption-aware noise from image representations under degraded conditions. Since only the TL-Denoiser is trained at test time, Dual-TTT is annotation-free and can be seamlessly integrated into existing TOOC models without modifying their original architecture. Extensive experiments on multiple recent TOOC baselines demonstrate the effectiveness of our method.

Problem

• Text-guided Open-vocabulary Object Counting
• robustness
• image corruption
• vision-language alignment
• adverse conditions

Innovation

• Test-Time Training
• Open-Vocabulary Object Counting
• Vision-Language Alignment
• Denoising Module
• Robustness Benchmark