AnyAD: Unified Any-Modality Anomaly Detection in Incomplete Multi-Sequence MRI

2025-12-24

AI Summary

To address poor generalizability in clinical brain MRI anomaly detection—caused by scarce abnormal annotations and frequent missing sequences across multi-modal acquisitions—this paper proposes the first modality-agnostic unified framework. Methodologically, we design a dual-path DINOv2 encoder integrated with statistical feature distribution alignment and an Intrinsic Normal Prototype (INP)-guided decoder; combined with random modality masking and indirect feature completion during training, our approach enables end-to-end anomaly detection and localization for arbitrary available modality combinations without retraining. Evaluated on three benchmarks—including BraTS2018—across seven modality configurations, our method consistently surpasses state-of-the-art methods, significantly enhancing robustness and clinical applicability under incomplete modality conditions. Key contributions include: (i) modality-agnostic representation learning, (ii) INP-driven anomaly amplification, and (iii) a cross-modal feature alignment paradigm.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

Reliable anomaly detection in brain MRI remains challenging due to the scarcity of annotated abnormal cases and the frequent absence of key imaging modalities in real clinical workflows. Existing single-class or multi-class anomaly detection (AD) models typically rely on fixed modality configurations, require repetitive training, or fail to generalize to unseen modality combinations, limiting their clinical scalability. In this work, we present a unified Any-Modality AD framework that performs robust anomaly detection and localization under arbitrary MRI modality availability. The framework integrates a dual-pathway DINOv2 encoder with a feature distribution alignment mechanism that statistically aligns incomplete-modality features with full-modality representations, enabling stable inference even with severe modality dropout. To further enhance semantic consistency, we introduce an Intrinsic Normal Prototypes (INPs) extractor and an INP-guided decoder that reconstruct only normal anatomical patterns while naturally amplifying abnormal deviations. Through randomized modality masking and indirect feature completion during training, the model learns to adapt to all modality configurations without re-training. Extensive experiments on BraTS2018, MU-Glioma-Post, and Pretreat-MetsToBrain-Masks demonstrate that our approach consistently surpasses state-of-the-art industrial and medical AD baselines across 7 modality combinations, achieving superior generalization. This study establishes a scalable paradigm for multimodal medical AD under real-world, imperfect modality conditions. Our source code is available at https://github.com/wuchangw/AnyAD.

Problem

• Detects brain anomalies with any MRI modality combination.
• Aligns incomplete modality features to full-modality representations.
• Generalizes to unseen modality sets without retraining.

Innovation

• Unified framework for any MRI modality anomaly detection
• Dual-pathway encoder with feature distribution alignment mechanism
• Intrinsic Normal Prototypes extractor and guided decoder