Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models

2025-08-05

AI Summary

In computational pathology, large tissue sections are segmented into multiple fragments and stitched into whole-mount slide (WMS) images; however, existing boundary-based stitching methods suffer from poor robustness due to tissue defects, deformations, staining heterogeneity, and edge abrasion. To address this, we propose the first semantic-driven stitching framework leveraging Vision Foundation Models (VFMs): it extracts high-dimensional latent semantic features from pretrained pathological VFMs, constructs cross-fragment semantic correspondence candidate sets, and enables robust pose estimation and precise spatial registration. By decoupling stitching from explicit geometric boundary constraints, our method significantly improves tolerance to nonrigid deformations and staining variations. Evaluated on three public histopathological datasets, it consistently outperforms state-of-the-art methods in boundary matching accuracy, yielding more stable and higher-fidelity WMS reconstructions.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Other Foundations of Machine Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

In histopathology, tissue samples are often larger than a standard microscope slide, making stitching of multiple fragments necessary to process entire structures such as tumors. Automated stitching is a prerequisite for scaling analysis, but is challenging due to possible tissue loss during preparation, inhomogeneous morphological distortion, staining inconsistencies, missing regions due to misalignment on the slide, or frayed tissue edges. This limits state-of-the-art stitching methods using boundary shape matching algorithms to reconstruct artificial whole mount slides (WMS). Here, we introduce SemanticStitcher using latent feature representations derived from a visual histopathology foundation model to identify neighboring areas in different fragments. Robust pose estimation based on a large number of semantic matching candidates derives a mosaic of multiple fragments to form the WMS. Experiments on three different histopathology datasets demonstrate that SemanticStitcher yields robust WMS mosaicing and consistently outperforms the state of the art in correct boundary matches.

Problem

• Automated stitching of large histopathology tissue fragments
• Overcoming challenges like tissue loss and distortion
• Improving accuracy in whole mount slide reconstruction

Innovation

• Uses visual histopathology foundation model features
• Robust pose estimation with semantic matches
• Outperforms state-of-the-art boundary matching methods