AdaSCALE: Adaptive Scaling for OOD Detection

2025-03-11

AI Summary

Existing out-of-distribution (OOD) detection methods rely on static percentile thresholds, compromising robust discrimination between in-distribution (ID) and OOD samples. To address this, we propose a fine-tuning-free, architecture-agnostic adaptive energy scaling mechanism. We first empirically observe that OOD samples exhibit significantly higher sensitivity to small input perturbations—manifested as larger magnitude shifts in activation responses—compared to ID samples. Leveraging this property, we introduce a sample-level dynamic threshold: the energy score’s percentile baseline is adaptively scaled according to each sample’s perturbation sensitivity. Our method requires only forward passes and is compatible with any pre-trained CNN or Transformer. Evaluated on ImageNet-1k across eight mainstream architectures, it achieves state-of-the-art performance: average false positive rate at 95% true positive rate (FPR@95) improves by 14.94 (near-OOD) and 21.67 (far-OOD) over the best prior baseline, OptFS.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

The ability of the deep learning model to recognize when a sample falls outside its learned distribution is critical for safe and reliable deployment. Recent state-of-the-art out-of-distribution (OOD) detection methods leverage activation shaping to improve the separation between in-distribution (ID) and OOD inputs. These approaches resort to sample-specific scaling but apply a static percentile threshold across all samples regardless of their nature, resulting in suboptimal ID-OOD separability. In this work, we propose extbf{AdaSCALE}, an adaptive scaling procedure that dynamically adjusts the percentile threshold based on a sample's estimated OOD likelihood. This estimation leverages our key observation: OOD samples exhibit significantly more pronounced activation shifts at high-magnitude activations under minor perturbation compared to ID samples. AdaSCALE enables stronger scaling for likely ID samples and weaker scaling for likely OOD samples, yielding highly separable energy scores. Our approach achieves state-of-the-art OOD detection performance, outperforming the latest rival OptFS by 14.94 in near-OOD and 21.67 in far-OOD datasets in average FPR@95 metric on the ImageNet-1k benchmark across eight diverse architectures. The code is available at: https://github.com/sudarshanregmi/AdaSCALE/

Problem

• Improves OOD detection by adaptive scaling
• Dynamically adjusts percentile threshold based on OOD likelihood
• Enhances ID-OOD separability using activation shifts

Innovation

• Dynamic percentile threshold adjustment
• Activation shift analysis for OOD likelihood
• Enhanced ID-OOD separability via adaptive scaling