DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup

2025-08-19

AI Summary

This work addresses the challenge of cross-category generalization without target-domain fine-tuning in few-shot anomaly segmentation. We propose DictAS, a dictionary-based universal visual anomaly detection framework. Methodologically, it constructs a transferable feature dictionary solely from a few normal reference images and performs anomaly localization within the vision-language model feature space via sparse dictionary lookup, query-discriminative regularization, contrastive query constraints, and text-alignment constraints. To our knowledge, this is the first work to introduce dictionary lookup into few-shot anomaly segmentation. It eliminates reliance on anomalous samples through self-supervised learning and enables zero-shot category generalization. Extensive experiments on seven public industrial and medical datasets demonstrate significant performance gains over state-of-the-art methods, validating its strong generalizability and practical utility.

Technology Category

• Computer Vision: Segmentation
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Recent vision-language models (e.g., CLIP) have demonstrated remarkable class-generalizable ability to unseen classes in few-shot anomaly segmentation (FSAS), leveraging supervised prompt learning or fine-tuning on seen classes. However, their cross-category generalization largely depends on prior knowledge of real seen anomaly samples. In this paper, we propose a novel framework, namely DictAS, which enables a unified model to detect visual anomalies in unseen object categories without any retraining on the target data, only employing a few normal reference images as visual prompts. The insight behind DictAS is to transfer dictionary lookup capabilities to the FSAS task for unseen classes via self-supervised learning, instead of merely memorizing the normal and abnormal feature patterns from the training set. Specifically, DictAS mainly consists of three components: (1) **Dictionary Construction** - to simulate the index and content of a real dictionary using features from normal reference images. (2) **Dictionary Lookup** - to retrieve queried region features from the dictionary via a sparse lookup strategy. When a query feature cannot be retrieved, it is classified as an anomaly. (3) **Query Discrimination Regularization**- to enhance anomaly discrimination by making abnormal features harder to retrieve from the dictionary. To achieve this, Contrastive Query Constraint and Text Alignment Constraint are further proposed. Extensive experiments on seven public industrial and medical datasets demonstrate that DictAS consistently outperforms state-of-the-art FSAS methods.

Problem

• Enables anomaly detection in unseen object categories without retraining
• Uses few normal reference images as visual prompts for generalization
• Transfers dictionary lookup capabilities to few-shot anomaly segmentation via self-supervised learning

Innovation

• Dictionary lookup for anomaly segmentation
• Self-supervised learning for unseen classes
• Sparse retrieval strategy detects anomalies