Energy-based Hopfield Boosting for Out-of-Distribution Detection

2024-05-14

arXiv.org

AI Summary

To address insufficient out-of-distribution (OOD) detection accuracy in open-world deployment, this paper proposes the first OOD detection method that integrates modern Hopfield energy dynamics into a boosting framework. The method leverages Hopfield energy to dynamically quantify sample proximity to the decision boundary, synergistically combining gradient-based weighted boosting with Outlier Exposure training to prioritize hard-to-classify anomalous samples near the boundary, and further employs confidence calibration to sharpen discriminative boundaries. Its core innovation lies in the first realization of Hopfield energy–driven selective sample reinforcement learning, substantially enhancing model sensitivity to unexpected OOD inputs. Evaluated on CIFAR-10 and CIFAR-100 benchmarks, the method achieves state-of-the-art FPR95 scores of 0.92% and 7.94%, respectively, setting new performance records for OOD detection.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Out-of-distribution (OOD) detection is critical when deploying machine learning models in the real world. Outlier exposure methods, which incorporate auxiliary outlier data in the training process, can drastically improve OOD detection performance compared to approaches without advanced training strategies. We introduce Hopfield Boosting, a boosting approach, which leverages modern Hopfield energy (MHE) to sharpen the decision boundary between the in-distribution and OOD data. Hopfield Boosting encourages the model to concentrate on hard-to-distinguish auxiliary outlier examples that lie close to the decision boundary between in-distribution and auxiliary outlier data. Our method achieves a new state-of-the-art in OOD detection with outlier exposure, improving the FPR95 metric from 2.28 to 0.92 on CIFAR-10 and from 11.76 to 7.94 on CIFAR-100.

Problem

• Out-Of-Distribution Detection
• Machine Learning Robustness
• Anomaly Recognition

Innovation

• Hopfield Boosting
• Out-of-Distribution Detection
• Energy Principle