Gradient Short-Circuit: Efficient Out-of-Distribution Detection via Feature Intervention

2025-07-02

AI Summary

This paper addresses out-of-distribution (OOD) detection for deep models in open-world settings, proposing a training-free inference-time intervention. The method exploits an intrinsic geometric property: in-feature-space local gradients of in-distribution (ID) samples exhibit high directional consistency, whereas those of OOD samples are highly divergent. To leverage this, the approach selectively shorts vulnerable feature coordinates prone to misclassification and suppresses OOD confidence scores; it further employs a first-order output approximation to avoid costly second forward passes. This work is the first to discover and utilize gradient directional consistency as an OOD signal, yielding a lightweight, plug-and-play solution requiring only minimal architectural modifications. Evaluated on standard OOD benchmarks, it significantly outperforms state-of-the-art methods while achieving superior accuracy, computational efficiency, and deployment practicality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Out-of-Distribution (OOD) detection is critical for safely deploying deep models in open-world environments, where inputs may lie outside the training distribution. During inference on a model trained exclusively with In-Distribution (ID) data, we observe a salient gradient phenomenon: around an ID sample, the local gradient directions for "enhancing" that sample's predicted class remain relatively consistent, whereas OOD samples--unseen in training--exhibit disorganized or conflicting gradient directions in the same neighborhood. Motivated by this observation, we propose an inference-stage technique to short-circuit those feature coordinates that spurious gradients exploit to inflate OOD confidence, while leaving ID classification largely intact. To circumvent the expense of recomputing the logits after this gradient short-circuit, we further introduce a local first-order approximation that accurately captures the post-modification outputs without a second forward pass. Experiments on standard OOD benchmarks show our approach yields substantial improvements. Moreover, the method is lightweight and requires minimal changes to the standard inference pipeline, offering a practical path toward robust OOD detection in real-world applications.

Problem

• Detect OOD samples via inconsistent gradient directions
• Short-circuit spurious gradients to reduce OOD confidence
• Approximate post-modification outputs without extra computation

Innovation

• Feature intervention for OOD detection
• Gradient short-circuit technique
• Local first-order approximation