On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain

2026-07-01

AI Summary

This study addresses the underexplored impact of structured expert pruning on factual reliability in high-stakes domains such as biomedicine, despite its potential to reduce deployment costs of Mixture-of-Experts (MoE) models. The authors present the first systematic evaluation of four MoE architectures combined with six structured pruning methods across varying sparsity levels, assessing both generative and classification tasks in biomedical and cross-domain settings. Results demonstrate that moderate pruning preserves task utility in biomedicine without substantially compromising factual accuracy, whereas aggressive pruning significantly elevates hallucination risks. In contrast, cross-domain scenarios exhibit rapid degradation in both utility and reliability under pruning. These findings underscore the critical influence of task and domain characteristics on safe model compression and highlight the inadequacy of relying solely on utility metrics for high-risk applications.

Abstract

Mixture-of-Experts (MoE) models offer inference speedups via selective activation but impose substantial memory requirements because the whole network must remain loaded. Structured expert pruning is a practical approach for reducing deployment costs in resource-constrained settings. However, prior studies primarily evaluate benchmark utility, leaving the effect of pruning on factual reliability underexplored, particularly in high-stakes domains such as biomedicine. In this paper, we investigate how domain-specific expert pruning affects both utility and reliability. We assess four MoE models, six pruning methods, and multiple pruning ratios across generation and classification tasks under in-domain (biomedical) and cross-domain settings. Results reveal that moderate pruning preserves in-domain utility without immediate reliability decline, although hallucination risks increase at extreme pruning ratios. When shifting to the general domain, both utility and reliability degrade rapidly. These findings indicate that safe compression depends heavily on the task and domain. Evaluating pruned MoE models solely on utility is inadequate for high-stakes deployment without reliability assessment.

Problem

• Mixture-of-Experts
• pruning
• factual reliability
• biomedical domain
• hallucination

Innovation

• Mixture-of-Experts
• structured pruning
• factual reliability
• biomedical domain
• hallucination risk