WISE-FUSE: Efficient Whole Slide Image Encoding via Coarse-to-Fine Patch Selection with VLM and LLM Knowledge Fusion

2025-08-20

AI Summary

Whole-slide images (WSIs) in computational pathology—often reaching gigapixel scale—require encoding tens of thousands to hundreds of thousands of high-resolution patches, resulting in encoding times spanning days to weeks, posing a critical bottleneck for clinical deployment. To address this, we propose an adaptive WSI encoding framework that synergistically integrates vision-language models (VLMs) and large language models (LLMs). First, a pathology-specific VLM generates descriptive text to guide coarse-grained identification of diagnostically salient regions. Second, knowledge distillation computes semantic similarity between low-resolution patches and class-descriptive text embeddings, enabling selective, fine-grained encoding of high-resolution patches. Finally, text embeddings are fused with visual features to enrich contextual representation. Experiments demonstrate that our method reduces WSI encoding time by over threefold while maintaining diagnostic accuracy on par with—or exceeding—that of exhaustive patch-level encoding baselines.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Whole slide images (WSIs) in computational pathology (CPath) pose a major computational challenge due to their gigapixel scale, often requiring the processing of tens to hundreds of thousands of high-resolution patches per slide. This results in prohibitive encoding costs, with preprocessing and training times extending to days or even weeks-making WSI encoding the most significant bottleneck in real-world deployment. In this work, we propose WISE-FUSE, an adaptive WSI encoding framework that leverages pathology-domain vision-language models and large language models to address this challenge by selectively processing diagnostically relevant regions. WISE-FUSE first computes similarity scores between low-resolution patches and class-specific textual descriptions using a knowledge distillation mechanism that preserves fine-grained diagnostic features. Based on these similarity scores, we select a small subset of informative regions for the target task, which quickly eliminates irrelevant patches at the coarse level. The corresponding high-resolution patches are then selectively encoded and fused with textual embeddings to reinforce diagnostic context. Extensive experiments demonstrate that WISE-FUSE reduces WSI encoding time by over threefold while achieving diagnostic performance comparable to or surpassing that of exhaustive patch processing, offering a scalable and practical solution for CPath.

Problem

• Reducing computational cost of gigapixel whole slide image encoding
• Selecting diagnostically relevant regions from thousands of patches
• Accelerating WSI preprocessing while maintaining diagnostic accuracy

Innovation

• Coarse-to-fine patch selection method
• Vision-language model similarity scoring
• Text-image embedding fusion for context