Zero-Shot Anomaly Detection with Dual-Branch Prompt Learning

2025-08-01

AI Summary

To address the weak generalization of zero-shot anomaly detection (ZSAD) under domain shift, this paper proposes a dual-branch prompt learning framework with label-free test-time adaptation. Methodologically: (1) a dual-branch prompt mechanism is designed, jointly leveraging a learnable prompt pool and structured semantic attribute encoding to enhance cross-domain semantic alignment; (2) a test-time adaptation strategy—requiring no human annotations—is introduced, wherein high-confidence pseudo-labels drive contrastive learning to dynamically optimize prompt parameters for target-domain distribution alignment. Extensive experiments across 13 industrial and medical benchmarks demonstrate significant improvements in both anomaly detection and localization performance, achieving state-of-the-art results. Notably, the method exhibits superior robustness and generalization under domain shift, outperforming existing approaches in challenging cross-domain ZSAD scenarios.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Zero-shot anomaly detection (ZSAD) enables identifying and localizing defects in unseen categories by relying solely on generalizable features rather than requiring any labeled examples of anomalies. However, existing ZSAD methods, whether using fixed or learned prompts, struggle under domain shifts because their training data are derived from limited training domains and fail to generalize to new distributions. In this paper, we introduce PILOT, a framework designed to overcome these challenges through two key innovations: (1) a novel dual-branch prompt learning mechanism that dynamically integrates a pool of learnable prompts with structured semantic attributes, enabling the model to adaptively weight the most relevant anomaly cues for each input image; and (2) a label-free test-time adaptation strategy that updates the learnable prompt parameters using high-confidence pseudo-labels from unlabeled test data. Extensive experiments on 13 industrial and medical benchmarks demonstrate that PILOT achieves state-of-the-art performance in both anomaly detection and localization under domain shift.

Problem

• Detect unseen anomalies without labeled examples
• Overcome domain shifts in zero-shot anomaly detection
• Improve generalization with dynamic prompt learning

Innovation

• Dual-branch prompt learning integrates learnable prompts
• Dynamic weighting of relevant anomaly cues per image
• Label-free test-time adaptation with pseudo-labels