Anomaly Detection by Clustering DINO Embeddings using a Dirichlet Process Mixture

2025-09-24

AI Summary

Unsupervised anomaly detection on large-scale medical imaging datasets is hindered by computational and memory bottlenecks arising from reliance on extensive normative feature banks. Method: We propose an efficient, scalable framework that extracts image embeddings using DINOv2 and models their distribution via a nonparametric Dirichlet Process Mixture Model (DPMM), which automatically determines the optimal number of clusters. Anomaly scores are computed as cosine similarities between ℓ²-normalized embeddings and cluster centroids, enabling direct generation of coarse-grained anomaly segmentation masks—eliminating the need for explicit memory banks. Contribution/Results: Our approach significantly reduces storage overhead, achieves state-of-the-art detection performance across multiple medical imaging benchmarks, accelerates inference by ≥50%, and enhances robustness through anatomically more consistent normalized embeddings.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Segmentation
• Machine Learning: Unsupervised & Self-Supervised Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this work, we leverage informative embeddings from foundational models for unsupervised anomaly detection in medical imaging. For small datasets, a memory-bank of normative features can directly be used for anomaly detection which has been demonstrated recently. However, this is unsuitable for large medical datasets as the computational burden increases substantially. Therefore, we propose to model the distribution of normative DINOv2 embeddings with a Dirichlet Process Mixture model (DPMM), a non-parametric mixture model that automatically adjusts the number of mixture components to the data at hand. Rather than using a memory bank, we use the similarity between the component centers and the embeddings as anomaly score function to create a coarse anomaly segmentation mask. Our experiments show that through DPMM embeddings of DINOv2, despite being trained on natural images, achieve very competitive anomaly detection performance on medical imaging benchmarks and can do this while at least halving the computation time at inference. Our analysis further indicates that normalized DINOv2 embeddings are generally more aligned with anatomical structures than unnormalized features, even in the presence of anomalies, making them great representations for anomaly detection. The code is available at https://github.com/NicoSchulthess/anomalydino-dpmm.

Problem

• Detect anomalies in medical imaging using DINO embeddings
• Overcome computational burden of memory banks for large datasets
• Model normative embeddings with Dirichlet Process Mixture model

Innovation

• Clustering DINO embeddings with Dirichlet Process Mixture
• Using component similarity as anomaly score function
• Leveraging normalized embeddings for anatomical alignment