A Convolutional Layer Activation Dimensionality Reduction for Out-of-Distribution and Adversarial Attack Detection Methods

2026-08-10

AI Summary

This work addresses the vulnerability of convolutional neural networks to out-of-distribution samples and adversarial attacks, stemming from a lack of reliable detection mechanisms. Existing detection methods based on intermediate activations struggle to balance information retention and computational efficiency during dimensionality reduction in convolutional layers. To overcome this limitation, the paper proposes a tunable compression-ratio method for reducing the dimensionality of convolutional activations, which can be seamlessly integrated into mainstream detection frameworks. The approach significantly lowers computational and memory overhead while preserving discriminative features essential for accurate detection. Extensive experiments demonstrate that the proposed method achieves state-of-the-art or comparable detection performance across multiple benchmarks, while enabling higher compression rates—thus offering an effective trade-off between efficiency and robustness.

Abstract

Despite the success of convolutional neural networks in image classification tasks and their general application in multi-modal models, their susceptibility to out-of-distribution and adversarial attack samples raises concerns regarding trustworthiness and safety. Among the approaches to tackle such issues, detection methods that analyze the model's intermediate activations to estimate a confidence score are a promising family that evaluates the decision process, relying on a dimensionality reduction step to enable efficient downstream processing of the high-dimensional activations. However, when considering convolutional layers, the dimensionality reduction methods in the literature either lack a mechanism to control the compression/information-loss trade-off or yield large representations. In this paper, we carefully analyze two state-of-the-art detection methods and their dimensionality reductions for convolutional layers and develop a novel reduction method with a controllable high-compression level. We extend these two state-of-the-art detection methods, enabling the usage of any dimensionality reduction, and evaluate their performance on out-of-distribution and adversarial attack detection. Results show that the detection methods with the proposed dimensionality reduction consistently perform better than, or comparable to, the strongest alternative. Furthermore, the proposed method is shown to reduce computation and memory footprints, given that it has the highest compression among the compared methods.

Problem

• out-of-distribution detection
• adversarial attack detection
• convolutional layer
• activation dimensionality reduction
• information-loss trade-off

Innovation

• dimensionality reduction
• convolutional activations
• out-of-distribution detection
• adversarial attack detection
• compression trade-off