TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

2026-03-09

AI Summary

This work addresses the challenge of simultaneously recognizing known classes and discovering novel ones when deploying offline-trained models in online data streams. To this end, we propose a test-time adaptive learning framework that enables end-to-end knowledge expansion through semantic-aware dynamic prototype updating and stable encoder fine-tuning. During the offline phase, a boundary-aware logit calibration mechanism is introduced to reserve embedding space for emerging classes. By abandoning conventional hash-based quantization in favor of continuous embedding representations, our approach effectively mitigates class fragmentation and explosion. Evaluated on standard Open Class Discovery (OCD) benchmarks, the proposed method significantly outperforms current state-of-the-art approaches, achieving substantial gains in both accuracy and stability for novel class discovery.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at \textcolor{blue}{https://github.com/ynanwu/TALON}.

Problem

• on-the-fly category discovery
• test-time adaptation
• category explosion
• feature quantization
• novel class discovery

Innovation

• test-time adaptation
• on-the-fly category discovery
• prototype update
• margin-aware calibration
• online learning