Spatial Transcriptomics as Images for Large-Scale Pretraining

2026-03-13

AI Summary

Existing pretraining methods for spatial transcriptomics are often limited by neglecting spatial dependencies or suffering from excessively high input dimensionality. To address these challenges, this work reformulates spatial transcriptomics data as multi-channel images and employs fixed-size local cropping to preserve spatial context while augmenting sample diversity. A gene subset selection strategy is introduced to effectively control input dimensionality. This approach establishes a unified data organization paradigm that simultaneously maintains spatial structure and enhances training efficiency. Evaluated across multiple downstream tasks, the proposed method significantly outperforms current pretraining approaches. Ablation studies further confirm the efficacy of both the spatial cropping strategy and the multi-channel image representation in capturing biologically relevant spatial patterns.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Spatial Transcriptomics (ST) profiles thousands of gene expression values at discrete spots with precise coordinates on tissue sections, preserving spatial context essential for clinical and pathological studies. With rising sequencing throughput and advancing platforms, the expanding data volumes motivate large-scale ST pretraining. However, the fundamental unit for pretraining, i.e., what constitutes a single training sample, remains ill-posed. Existing choices fall into two camps: (1) treating each spot as an independent sample, which discards spatial dependencies and collapses ST into single-cell transcriptomics; and (2) treating an entire slide as a single sample, which produces prohibitively large inputs and drastically fewer training examples, undermining effective pretraining. To address this gap, we propose treating spatial transcriptomics as croppable images. Specifically, we define a multi-channel image representation with fixed spatial size by cropping patches from raw slides, thereby preserving spatial context while substantially increasing the number of training samples. Along the channel dimension, we define gene subset selection rules to control input dimensionality and improve pretraining stability. Extensive experiments show that the proposed image-like dataset construction for ST pretraining consistently improves downstream performance, outperforming conventional pretraining schemes. Ablation studies verify that both spatial patching and channel design are necessary, establishing a unified, practical paradigm for organizing ST data and enabling large-scale pretraining.

Problem

• Spatial Transcriptomics
• pretraining
• training sample
• spatial context
• data representation

Innovation

• Spatial Transcriptomics
• Image-based Pretraining
• Patch Cropping
• Multi-channel Representation
• Large-scale Pretraining