MODE: Learning compositional representations of complex systems with Mixtures Of Dynamical Experts

2025-10-10

AI Summary

Dynamic systems in life sciences often exhibit regime-switching behaviors—such as cell fate transitions from proliferation to differentiation—characterized by multimodality, nonsmoothness, and heavy noise, rendering conventional flow-based models ineffective due to their reliance on local smoothness assumptions. To address this, we propose the Dynamic Mixture of Experts (D-MoE) framework: a sparse, interpretable dynamical decomposition architecture leveraging neural gating for regime-aware routing, integrated with graph-enhanced neural ODEs and variational latent modeling to enable end-to-end training under low-sample, high-noise conditions. D-MoE is the first method to achieve unsupervised discovery and long-term accurate prediction of proliferation/differentiation fates at single-cell resolution. It significantly outperforms state-of-the-art approaches on synthetic benchmarks, cell-cycle simulations, and branching developmental trajectories, and successfully identifies and forecasts critical fate-decision events in human scRNA-seq data.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Dynamical systems in the life sciences are often composed of complex mixtures of overlapping behavioral regimes. Cellular subpopulations may shift from cycling to equilibrium dynamics or branch towards different developmental fates. The transitions between these regimes can appear noisy and irregular, posing a serious challenge to traditional, flow-based modeling techniques which assume locally smooth dynamics. To address this challenge, we propose MODE (Mixture Of Dynamical Experts), a graphical modeling framework whose neural gating mechanism decomposes complex dynamics into sparse, interpretable components, enabling both the unsupervised discovery of behavioral regimes and accurate long-term forecasting across regime transitions. Crucially, because agents in our framework can jump to different governing laws, MODE is especially tailored to the aforementioned noisy transitions. We evaluate our method on a battery of synthetic and real datasets from computational biology. First, we systematically benchmark MODE on an unsupervised classification task using synthetic dynamical snapshot data, including in noisy, few-sample settings. Next, we show how MODE succeeds on challenging forecasting tasks which simulate key cycling and branching processes in cell biology. Finally, we deploy our method on human, single-cell RNA sequencing data and show that it can not only distinguish proliferation from differentiation dynamics but also predict when cells will commit to their ultimate fate, a key outstanding challenge in computational biology.

Problem

• Modeling noisy transitions between overlapping behavioral regimes
• Decomposing complex dynamics into interpretable sparse components
• Predicting cell fate commitment from single-cell RNA sequencing data

Innovation

• Mixture Of Dynamical Experts decomposes dynamics into interpretable components
• Neural gating mechanism enables unsupervised discovery of behavioral regimes
• Agents can jump between governing laws to handle noisy transitions