Out-of-distribution evaluations of channel agnostic masked autoencoders in fluorescence microscopy

2025-03-24

AI Summary

In high-throughput fluorescence microscopy screening, visual model generalization is severely hampered by multiple concurrent distribution shifts—including experimental batches, perturbations, fluorescent labels, and cell types. Existing transfer evaluation methods fail to disentangle these distinct shift sources, impeding principled model design. To address this, we propose the first decoupled distribution shift evaluation framework, enabling independent quantification of each shift’s impact on the JUMP-CP benchmark. We further introduce Campfire, a channel-agnostic masked autoencoder that employs a shared decoder to jointly model multi-channel fluorescence signals—thereby overcoming scalability and cross-domain generalization bottlenecks in multi-label settings. Extensive experiments demonstrate that Campfire significantly outperforms channel-specific models across batch-, perturbation-, label-, and cell-type-transfer tasks, validating its robustness to biological experimental variability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Developing computer vision for high-content screening is challenging due to various sources of distribution-shift caused by changes in experimental conditions, perturbagens, and fluorescent markers. The impact of different sources of distribution-shift are confounded in typical evaluations of models based on transfer learning, which limits interpretations of how changes to model design and training affect generalisation. We propose an evaluation scheme that isolates sources of distribution-shift using the JUMP-CP dataset, allowing researchers to evaluate generalisation with respect to specific sources of distribution-shift. We then present a channel-agnostic masked autoencoder $mathbf{Campfire}$ which, via a shared decoder for all channels, scales effectively to datasets containing many different fluorescent markers, and show that it generalises to out-of-distribution experimental batches, perturbagens, and fluorescent markers, and also demonstrates successful transfer learning from one cell type to another.

Problem

• Evaluates generalization of vision models under distribution-shift in microscopy
• Proposes method to isolate distribution-shift sources for clearer model assessment
• Develops channel-agnostic autoencoder for robust fluorescence marker adaptation

Innovation

• Channel-agnostic masked autoencoder for fluorescence microscopy
• Isolated distribution-shift evaluation using JUMP-CP dataset
• Shared decoder for multi-channel fluorescent markers