No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology

2025-01-31

AI Summary

Current deep learning models for gene expression prediction overlook critical distinctions in cellular states (e.g., differentiated vs. undifferentiated) and experimental generation mechanisms, implicitly imposing biologically implausible assumptions that impair generalizability and interpretability. To address this, we propose a semi-mechanistic modeling framework specifically designed for gene expression prediction. Our approach unifies *in vitro* and *in vivo* CRISPR perturbation data, explicitly incorporating experimental design, biological constraints, and structural causal relationships. We theoretically analyze the limitations of prevailing methods, establishing formal connections to variational autoencoders and structural causal models. Furthermore, we introduce a mechanism-driven correction loss and an error-guided dynamic batching strategy. Experiments demonstrate substantial improvements in predictive accuracy, provide interpretable error溯源 tools for root-cause analysis, and inform high-throughput screening by optimizing experimental batch design and data generation strategies.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Despite substantial efforts, deep learning has not yet delivered a transformative impact on elucidating regulatory biology, particularly in the realm of predicting gene expression profiles. Here, we argue that genuine"foundation models"of regulatory biology will remain out of reach unless guided by frameworks that integrate mechanistic insight with principled experimental design. We present one such ground-up, semi-mechanistic framework that unifies perturbation-based experimental designs across both in vitro and in vivo CRISPR screens, accounting for differentiating and non-differentiating cellular systems. By revealing previously unrecognised assumptions in published machine learning methods, our approach clarifies links with popular techniques such as variational autoencoders and structural causal models. In practice, this framework suggests a modified loss function that we demonstrate can improve predictive performance, and further suggests an error analysis that informs batching strategies. Ultimately, since cellular regulation emerges from innumerable interactions amongst largely uncharted molecular components, we contend that systems-level understanding cannot be achieved through structural biology alone. Instead, we argue that real progress will require a first-principles perspective on how experiments capture biological phenomena, how data are generated, and how these processes can be reflected in more faithful modelling architectures.

Problem

• Deep Learning Limitations
• Gene Expression Prediction
• Cell Differentiation Modeling

Innovation

• Integrated Experimental Design and Machine Learning
• Differential and Undifferential Cell State Consideration
• Enhanced Gene Expression Prediction and Model Understanding