Logit Scaling for Out-of-Distribution Detection

2024-09-02

arXiv.org

AI Summary

Existing out-of-distribution (OOD) detection methods for open-world settings often rely on training-data statistics or model fine-tuning, limiting generalizability and practicality. To address this, we propose Logit Temperature Scaling (LTS), the first purely post-hoc, architecture-agnostic logit scaling mechanism. LTS operates solely on the pre-softmax logits—applying either learnable or heuristic scaling—to enhance separation between in-distribution (ID) and OOD confidence scores. It requires no access to training data, modifies neither network architecture nor parameters, and introduces zero additional training overhead. Evaluated across CIFAR, ImageNet, and OpenOOD benchmarks—including 3 ID datasets, 14 OOD datasets, and 9 mainstream architectures—LTS consistently outperforms existing unsupervised OOD detection approaches, achieving state-of-the-art performance while demonstrating strong cross-architecture robustness.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Computer Vision: Object Detection & Categorization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The safe deployment of machine learning and AI models in open-world settings hinges critically on the ability to detect out-of-distribution (OOD) data accurately, data samples that contrast vastly from what the model was trained with. Current approaches to OOD detection often require further training the model, and/or statistics about the training data which may no longer be accessible. Additionally, many existing OOD detection methods struggle to maintain performance when transferred across different architectures. Our research tackles these issues by proposing a simple, post-hoc method that does not require access to the training data distribution, keeps a trained network intact, and holds strong performance across a variety of architectures. Our method, Logit Scaling (LTS), as the name suggests, simply scales the logits in a manner that effectively distinguishes between in-distribution (ID) and OOD samples. We tested our method on benchmarks across various scales, including CIFAR-10, CIFAR-100, ImageNet and OpenOOD. The experiments cover 3 ID and 14 OOD datasets, as well as 9 model architectures. Overall, we demonstrate state-of-the-art performance, robustness and adaptability across different architectures, paving the way towards a universally applicable solution for advanced OOD detection.

Problem

• Detects out-of-distribution data without retraining models
• Maintains performance across diverse model architectures
• Eliminates need for training data distribution access

Innovation

• Post-hoc method without training data access
• Scales logits to distinguish ID and OOD
• Maintains performance across diverse architectures