HiST: A Hierarchical Sparse Transformer for Cross-Modal Spatial Transcriptomics Modeling

2026-06-12

AI Summary

This work addresses the challenge of efficiently inferring sparsely and irregularly distributed spatial transcriptomic expression from standard H&E-stained histology slides. The authors propose HiST, a hierarchical sparse Transformer architecture that captures local geometric correspondences through sparse window-based attention while integrating multi-scale contextual information. HiST incorporates a location-indexed sparse field modeling strategy, a resolution-adaptive encoder–decoder design, and slide-calibration tokens to enable global conditional modulation, effectively mitigating inter-slide heterogeneity. Evaluated across multi-organ, multi-source benchmarks, HiST substantially outperforms existing methods, achieving higher prediction accuracy while significantly reducing computational cost and memory consumption.

Abstract

Spatial transcriptomics (ST) links gene expression with tissue morphology but remains expensive and low-throughput, motivating surrogates that infer expression from routine histology. Whole-slide H&E-to-ST inference pairs a gigapixel image with gene measurements at a sparse, irregular set of locations, making multiscale modeling challenging without incurring dense-grid overhead or quadratic token mixing. We propose HiST, a hierarchical sparse transformer that treats measured locations as a lattice-indexed sparse field and builds a dyadic encoder--decoder directly on the active tissue footprint. HiST combines sparse window attention for local geometric correspondence with resolution-changing operators for rapid multiscale context integration. For a fixed window size, the dominant runtime and memory scale with the number of observed locations rather than the dense slide area. To mitigate slide-specific acquisition variation, HiST adds a bottlenecked global conditioning pathway via a \emph{slide calibration token} that summarizes slide-level context and conditions local representations. On a multi-organ benchmark spanning diverse tissues and acquisition sources, HiST improves predictive performance over recent baselines while reducing runtime and peak memory.

Problem

• spatial transcriptomics
• cross-modal inference
• sparse modeling
• multiscale modeling
• histology-to-transcriptomics

Innovation

• hierarchical sparse transformer
• spatial transcriptomics
• sparse window attention
• multiscale modeling
• slide calibration token