VNDUQE: Information-Theoretic Novelty Detection using Deep Variational Information Bottleneck

2026-05-12

AI Summary

This work addresses the unreliability of out-of-distribution (OOD) detection and the lack of theoretical grounding in uncertainty estimation for neural networks in safety-critical applications. To this end, the authors propose a novel approach based on the deep variational information bottleneck (VIB), which constrains information flow in learned representations and integrates information-theoretic measures—specifically KL divergence and predictive entropy—into OOD detection for the first time. This integration enables a parallel detection strategy that substantially outperforms the maximum softmax probability (MSP) baseline in both far- and near-OOD settings. On MNIST, the method achieves 95.3% AUROC and 92% true positive rate at a 5% false positive rate, representing a 32-percentage-point improvement. Additionally, the induced information compression reduces expected calibration error by 38%, significantly enhancing model calibration.

Abstract

Detecting out-of-distribution (OOD) samples is critical for safe deployment of neural networks in safety-critical applications. While maximum softmax probability (MSP) provides a simple baseline, it lacks theoretical grounding and suffers from miscalibration. We propose VNDUQE (VIB-based Novelty Detection and Uncertainty Quantification for Nondestructive Evaluation), which investigates novelty detection through the Deep Variational Information Bottleneck (VIB), which explicitly constrains information flow through learned representations. We train VIB models on MNIST with held-out digit classes and evaluate OOD detection using information-theoretic metrics: KL divergence and prediction entropy. Our results reveal complementary detection signals: KL divergence achieves perfect detection (100\% AUROC on noise) on far-OOD samples (noise, domain shift), while prediction entropy excels at near-OOD detection (94.7\% AUROC on novel digit classes). A parallel detection strategy combining both metrics achieves 95.3\% average AUROC and 92\% true positive rate at 5\% false positive rate, which is a 32 percentage point improvement over baseline MSP (85.0\% AUROC, 60.1\% TPR). Compression via the information bottleneck principle ($β=10^{-3}$) reduces Expected Calibration Error by 38\%, demonstrating that information-theoretic constraints produce fundamentally more reliable uncertainty estimates. These findings directly support active learning with expensive computational oracles, where well-calibrated novelty detection enables principled threshold selection for oracle queries.

Problem

• out-of-distribution detection
• novelty detection
• uncertainty quantification
• safe deployment
• neural networks

Innovation

• Variational Information Bottleneck
• Out-of-Distribution Detection
• Information-Theoretic Uncertainty
• Novelty Detection
• Uncertainty Quantification