Weight Space Correlation Analysis: Quantifying Feature Utilization in Deep Learning Models

2025-12-15

AI Summary

Medical imaging deep learning models are prone to shortcut learning, inadvertently or deliberately exploiting confounding metadata—such as scanner manufacturer—to degrade predictive reliability. To address this, we propose a weight-space correlation analysis method that quantifies, for the first time, a model’s *actual reliance* on confounders embedded in its representations—using cosine similarity among classification head weight vectors as an interpretable, post-hoc metric—rather than merely detecting their presence. Our approach leverages a multi-task projection framework to assess a model’s intrinsic ability to disentangle acquisition-invariant features under unbiased training conditions. Empirical evaluation on the SA-SonoNet architecture for spontaneous preterm birth (sPTB) prediction demonstrates that learned weights exhibit significant correlations with clinically meaningful variables (e.g., birth weight) while remaining decoupled from scanner-related metadata. This work introduces the first quantitative, interpretable diagnostic tool for shortcut learning in medical AI, advancing model trustworthiness and clinical deployability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Deep learning models in medical imaging are susceptible to shortcut learning, relying on confounding metadata (e.g., scanner model) that is often encoded in image embeddings. The crucial question is whether the model actively utilizes this encoded information for its final prediction. We introduce Weight Space Correlation Analysis, an interpretable methodology that quantifies feature utilization by measuring the alignment between the classification heads of a primary clinical task and auxiliary metadata tasks. We first validate our method by successfully detecting artificially induced shortcut learning. We then apply it to probe the feature utilization of an SA-SonoNet model trained for Spontaneous Preterm Birth (sPTB) prediction. Our analysis confirmed that while the embeddings contain substantial metadata, the sPTB classifier's weight vectors were highly correlated with clinically relevant factors (e.g., birth weight) but decoupled from clinically irrelevant acquisition factors (e.g. scanner). Our methodology provides a tool to verify model trustworthiness, demonstrating that, in the absence of induced bias, the clinical model selectively utilizes features related to the genuine clinical signal.

Problem

• Detects shortcut learning in medical imaging models
• Quantifies feature utilization via weight space correlation
• Verifies model trustworthiness by analyzing feature selectivity

Innovation

• Weight Space Correlation Analysis quantifies feature utilization alignment
• Method detects shortcut learning by analyzing classification head correlations
• Verifies model trustworthiness by decoupling clinical from irrelevant factors