Causal integration of chemical structures improves representations of microscopy images for morphological profiling

2025-04-13

AI Summary

Current high-throughput microscopy image analysis predominantly relies on unimodal visual features, neglecting causal intervention information—such as chemical perturbations—leading to insufficient robustness in cellular phenotypic representation. To address this, we propose MICON, a molecular-image contrastive learning framework that, for the first time, models chemical structures as causal intervention variables. MICON integrates graph neural networks (GNNs) to encode molecular structures and convolutional neural networks (CNNs) or vision transformers (ViTs) to extract image features, while introducing a counterfactual loss to enforce causal alignment between modalities. Departing from conventional multimodal fusion paradigms, MICON achieves superior performance under rigorous cross-center and cross-batch evaluation, significantly outperforming CellProfiler and state-of-the-art deep representation methods. It improves consistency in drug effect identification by 27% and substantially enhances generalizability and reproducibility.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent advances in self-supervised deep learning have improved our ability to quantify cellular morphological changes in high-throughput microscopy screens, a process known as morphological profiling. However, most current methods only learn from images, despite many screens being inherently multimodal, as they involve both a chemical or genetic perturbation as well as an image-based readout. We hypothesized that incorporating chemical compound structure during self-supervised pre-training could improve learned representations of images in high-throughput microscopy screens. We introduce a representation learning framework, MICON (Molecular-Image Contrastive Learning), that models chemical compounds as treatments that induce counterfactual transformations of cell phenotypes. MICON significantly outperforms classical hand-crafted features such as CellProfiler and existing deep-learning-based representation learning methods in challenging evaluation settings where models must identify reproducible effects of drugs across independent replicates and data-generating centers. We demonstrate that incorporating chemical compound information into the learning process provides consistent improvements in our evaluation setting and that modeling compounds specifically as treatments in a causal framework outperforms approaches that directly align images and compounds in a single representation space. Our findings point to a new direction for representation learning in morphological profiling, suggesting that methods should explicitly account for the multimodal nature of microscopy screening data.

Problem

• Improving image representations in microscopy via chemical structure integration
• Enhancing morphological profiling with multimodal self-supervised learning
• Modeling drug effects as counterfactual phenotype transformations causally

Innovation

• Integrates chemical structures with microscopy images
• Uses self-supervised contrastive learning framework
• Models compounds as causal phenotype treatments