Input-Dependent Fisher Information for Local Sensitivity Analysis of Medical Image Classifiers

2026-06-15

AI Summary

This work addresses the limited intrinsic interpretability of existing medical image classifiers regarding prediction sensitivity and the weak alignment between post-hoc explanation methods and model prediction distributions. To overcome these limitations, the study introduces, for the first time, the input-dependent Fisher Information Matrix (iFIM) for local sensitivity analysis in medical image classification. By efficiently approximating iFIM via a Gram matrix and extracting its dominant eigensubspace, the method decomposes input images into high-sensitivity and orthogonal components, thereby intrinsically characterizing the model’s response to perturbations. Experiments demonstrate that the high-sensitivity iFIM components are strongly correlated with prediction confidence and variations in classification performance, validating the approach as an effective and superior tool for sensitivity analysis across diverse medical imaging tasks and model architectures.

Abstract

Deep neural networks have achieved strong performance in medical image classification, but often work like black-box. Commonly used post-hoc interpretation methods often provide heuristic visualizations whose relationship to the classifier's predictive distribution is indirect. This work introduces a local sensitivity analysis framework based on the input-dependent Fisher Information Matrix (iFIM) of a trained classifier. The iFIM characterizes how the classifier's predictive distribution changes under infinitesimal perturbations of the input image. By using a Gram-matrix formulation, the nonzero eigenspectrum of the iFIM can be recovered without explicitly forming the full image-dimensional Fisher matrix. The leading iFIM eigenspace is then used to project an input image into a high local-sensitivity component and its orthogonal component. These components provide a model-intrinsic description of local predictive sensitivity, rather than a conventional pixel-wise attribution heatmap or a causal segmentation of task-relevant anatomy. The framework is evaluated on controlled and clinical medical image classification tasks using multiple classifier architectures. Perturbation-based experiments show that high-sensitivity iFIM components are more strongly coupled to changes in predictive confidence and classification performance than lower-sensitivity complementary components. The results support the iFIM framework as a principled tool for analyzing local decision sensitivity and for complementing existing attribution-based interpretability methods in medical imaging.

Problem

• medical image classification
• black-box models
• interpretability
• local sensitivity analysis
• predictive distribution

Innovation

• input-dependent Fisher Information Matrix
• local sensitivity analysis
• medical image classification
• model interpretability
• Gram matrix eigenspectrum