Vendi Novelty Scores for Out-of-Distribution Detection

2026-02-10

AI Summary

This work addresses the challenge of out-of-distribution (OOD) detection in machine learning systems by proposing a novel method based on the Vendi Score, introducing diversity quantification into OOD detection for the first time. The approach measures the incremental impact of a test sample on the diversity of the training feature set—termed the Vendi Novelty Score (VNS)—to quantify sample novelty in a non-parametric manner with linear time complexity. It avoids explicit density modeling and effectively integrates both local and global novelty signals. Extensive experiments across multiple image classification benchmarks and neural architectures demonstrate that the method achieves state-of-the-art performance, maintaining strong detection capability even when trained on as little as 1% of the original training data.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Object Detection & Categorization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Out-of-distribution (OOD) detection is critical for the safe deployment of machine learning systems. Existing post-hoc detectors typically rely on model confidence scores or likelihood estimates in feature space, often under restrictive distributional assumptions. In this work, we introduce a third paradigm and formulate OOD detection from a diversity perspective. We propose the Vendi Novelty Score (VNS), an OOD detector based on the Vendi Scores (VS), a family of similarity-based diversity metrics. VNS quantifies how much a test sample increases the VS of the in-distribution feature set, providing a principled notion of novelty that does not require density modeling. VNS is linear-time, non-parametric, and naturally combines class-conditional (local) and dataset-level (global) novelty signals. Across multiple image classification benchmarks and network architectures, VNS achieves state-of-the-art OOD detection performance. Remarkably, VNS retains this performance when computed using only 1% of the training data, enabling deployment in memory- or access-constrained settings.

Problem

• out-of-distribution detection
• novelty detection
• machine learning safety
• distributional shift
• anomaly detection

Innovation

• Vendi Novelty Score
• out-of-distribution detection
• diversity-based detection
• non-parametric method
• linear-time algorithm