Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassifications

2025-12-22

AI Summary

Deep neural models suffer from opacity, low trustworthiness, and fragmented post-hoc robustness analysis in high-stakes applications: prevailing confidence estimation methods require model retraining, while out-of-distribution (OOD) detection, adversarial example identification, and in-distribution (ID) misclassification localization are typically addressed in isolation. This paper proposes MACS—a lightweight, plug-and-play, post-hoc analysis framework that requires no retraining. MACS constructs a classification graph by interpreting multi-layer activations of pretrained models to jointly enable confidence scoring, OOD detection, adversarial sample identification, and ID misclassification localization. Its core innovation lies in unifying these three critical robustness tasks within a single, efficient confidence scoring mechanism—integrating inter-layer consistency quantification with multi-level feature aggregation. Evaluated on VGG16 and ViT-B16, MACS surpasses state-of-the-art methods across all tasks, improving detection accuracy by 3.2–7.8% while incurring only 12–18% of the computational overhead of competing approaches.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The recent explosive growth in Deep Neural Networks applications raises concerns about the black-box usage of such models, with limited trasparency and trustworthiness in high-stakes domains, which have been crystallized as regulatory requirements such as the European Union Artificial Intelligence Act. While models with embedded confidence metrics have been proposed, such approaches cannot be applied to already existing models without retraining, limiting their broad application. On the other hand, post-hoc methods, which evaluate pre-trained models, focus on solving problems related to improving the confidence in the model's predictions, and detecting Out-Of-Distribution or Adversarial Attacks samples as independent applications. To tackle the limited applicability of already existing methods, we introduce Multi-Layer Analysis for Confidence Scoring (MACS), a unified post-hoc framework that analyzes intermediate activations to produce classification-maps. From the classification-maps, we derive a score applicable for confidence estimation, detecting distributional shifts and adversarial attacks, unifying the three problems in a common framework, and achieving performances that surpass the state-of-the-art approaches in our experiments with the VGG16 and ViTb16 models with a fraction of their computational overhead.

Problem

• Detects out-of-distribution samples and adversarial attacks
• Estimates confidence in predictions without retraining models
• Unifies multiple reliability issues in a single framework

Innovation

• Multi-layer activation analysis for unified confidence scoring
• Post-hoc framework without retraining existing models
• Classification-maps detect distribution shifts and adversarial attacks