KHiM-Mamba: Injecting Pathology Knowledge into Mamba via Hidden-State Modulation for Whole Slide Image Analysis

2026-08-14

AI Summary

This study addresses the issue of attention misallocation in existing State Space Models (SSMs) for Whole Slide Image (WSI) analysis caused by purely vision-driven mechanisms. To overcome this, we propose KHiM-Mamba, which introduces a novel knowledge-aware hidden state modulation mechanism that integrates pathological priors directly into the core SSM layers. Furthermore, the method leverages Large Language Model-based semantic retrieval to guide adaptive evidence accumulation, effectively mitigating noise interference in long-sequence modeling. Extensive experiments demonstrate that KHiM-Mamba achieves state-of-the-art performance across four tasks and eleven benchmarks. These results validate the effectiveness of knowledge-guided state space models in enabling precise diagnosis from whole slide images, marking a significant advancement in computational pathology.

Abstract

Whole slide image analysis is commonly formulated as multiple instance learning (MIL), where instance features are contextually updated and aggregated into a slide representation, a process we term slide encoding dynamics. Recently, selective state-space models (SSM) have emerged as promising MIL architectures due to their long-sequence modeling capability and linear complexity. However, existing SSM-based MIL methods rely solely on visual features during MIL. Meanwhile, in large-scale WSIs, where sparse diagnostically decisive regions are surrounded by abundant irrelevant information, such purely vision-driven selective dynamics can misallocate state updates and readouts, causing the evolving SSM state to accumulate task-irrelevant evidence and dilute critical diagnostic cues over long scan trajectories. In this work, we propose the Knowledge-Aware Hidden-State Modulation architecture (KHiM-Mamba), which innovatively regulates Mamba's core selective state-space mechanism with explicit knowledge priors, steering slide encoding dynamics toward diagnostically meaningful evidence accumulation. Specifically, we redesign the original SSM layer to perform knowledge modulation operations during the evolution of hidden states, thereby guiding what visual evidence is accumulated and retrieved from the hidden state at each encoding step. Furthermore, we additionally introduce a local-adaptive vocabulary retrieval module that uses large language models to assign each patch fine-grained, tissue-specific semantic descriptions, enabling precise modulation across diverse tasks. Experiments on 11 public benchmarks across 4 tasks show that KHiM-Mamba consistently achieves state-of-the-art performance.

Problem

• Whole Slide Image Analysis
• Multiple Instance Learning
• Selective State-Space Models
• Slide Encoding Dynamics
• Pathology Knowledge

Innovation

• Knowledge-Aware Hidden-State Modulation
• Selective State-Space Models
• Whole Slide Image Analysis
• Local-Adaptive Vocabulary Retrieval
• Multiple Instance Learning