Gated Spatial Redundancy Projection for Pathology Transformer Attentions

2026-08-08

AI Summary

This work addresses the challenge posed by high spatial redundancy among neighboring patches in whole-slide pathological images, which can cause Transformer self-attention mechanisms to over-mix local features and obscure critical diagnostic signals. To mitigate this, the authors propose Gated SRP, a lightweight, plug-and-play module that estimates a local redundancy direction for each patch and attention head within the self-attention layer, projects the output onto this direction, and applies a learnable sign-gated geometric correction. This is the first attention modulation mechanism explicitly designed for the spatial redundancy inherent in histopathology images, introducing only a 0.02% parameter overhead while effectively preserving discriminative information. Experiments demonstrate that the method achieves the highest average C-index across five TCGA survival cohorts, outperforms baselines on 12 of 16 metrics across five slide-level classification datasets, and attains state-of-the-art AUC on three of them.

Abstract

Transformer models are increasingly used for whole-slide image analysis in computational pathology. Yet, WSIs differ fundamentally from natural images: neighbouring patches often contain highly similar tissue type, stain, texture, and cellular composition. We identify this local spatial redundancy as a pathology-specific failure mode of self-attention, where dominant neighbourhood features can be repeatedly mixed into patch-tokens and weaken subtle diagnostic or prognostic deviations. We propose Gated Spatial Redundancy Projection (Gated SRP), a lightweight drop-in correction module for self-attention layers. For each patch token and attention head, Gated SRP estimates a local redundancy axis from neighbouring value vectors, projects the attention output onto this axis, and applies a learned signed gate to correct the redundancy-aligned component geometrically. Across five TCGA survival cohorts, Gated SRP obtains the highest mean C-index among the compared attention variants in all cohorts, with an average improvement over the base attention, while adding only +0.02% parameters. Across five slide-level classification datasets, it improves the base attention on 12 of 16 reported metrics and achieves the best AUC on three datasets. Code is publicly available at https://github.com/AtlasAnalyticsLab/GatedSRP.

Problem

• spatial redundancy
• self-attention
• computational pathology
• whole-slide image
• diagnostic deviation

Innovation

• Gated Spatial Redundancy Projection
• self-attention
• computational pathology
• whole-slide image
• spatial redundancy