Mixture-of-experts Wishart model for covariance matrices with an application to Cancer drug screening

2026-02-14

AI Summary

This work proposes a Mixture-of-Experts Wishart model (MoE-Wishart) to address the challenge of modeling heterogeneity in covariance matrices under covariate dependence. For the first time, the mixture-of-experts architecture is integrated into a Wishart mixture model, where a multinomial logistic gating network allows component weights to vary smoothly with covariates, thereby enabling nonlinear characterization of heterogeneous covariance structures and adaptive subgroup identification. Inference combines Bayesian and maximum likelihood frameworks, leveraging a Gibbs-within-Metropolis-Hastings scheme alongside the EM algorithm to effectively accommodate the geometry of the Wishart likelihood and the gating network. Experiments demonstrate that the model accurately recovers latent subgroups and their associated covariance structures in simulated data, and successfully uncovers complex covariance patterns between drug dosage and repeated measurements in cancer drug sensitivity analysis.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Covariance matrices arise naturally in different scientific fields, including finance, genomics, and neuroscience, where they encode dependence structures and reveal essential features of complex multivariate systems. In this work, we introduce a comprehensive Bayesian framework for analyzing heterogeneous covariance data through both classical mixture models and a novel mixture-of-experts Wishart (MoE-Wishart) model. The proposed MoE-Wishart model extends standard Wishart mixtures by allowing mixture weights to depend on predictors through a multinomial logistic gating network. This formulation enables the model to capture complex, nonlinear heterogeneity in covariance structures and to adapt subpopulation membership probabilities to covariate-dependent patterns. To perform inference, we develop an efficient Gibbs-within-Metropolis-Hastings sampling algorithm tailored to the geometry of the Wishart likelihood and the gating network. We additionally derive an Expectation-Maximization algorithm for maximum likelihood estimation in the mixture-of-experts setting. Extensive simulation studies demonstrate that the proposed Bayesian and maximum likelihood estimators achieve accurate subpopulation recovery and estimation under a range of heterogeneous covariance scenarios. Finally, we present an innovative application of our methodology to a challenging dataset: cancer drug sensitivity profiles, illustrating the ability of the MoE-Wishart model to leverage covariance across drug dosages and replicate measurements. Our methods are implemented in the \texttt{R} package \texttt{moewishart} available at https://github.com/zhizuio/moewishart .

Problem

• covariance matrices
• heterogeneity
• mixture-of-experts
• Wishart model
• drug sensitivity

Innovation

• Mixture-of-experts
• Wishart model
• covariance heterogeneity
• Bayesian inference
• gating network