QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis

2026-04-07

AI Summary

This work addresses the challenges of multimodal sentiment analysis in real-world scenarios, where dynamic noise and missing modalities hinder performance, and existing methods struggle to adapt to continuously varying modality reliability. The authors propose a continuous reliability spectrum to jointly model both modality absence and quality degradation, and introduce a Quality-Aware Mixture-of-Experts (QA-MoE) framework. Within this framework, self-supervised aleatoric uncertainty quantifies modality reliability in a fine-grained, continuous manner, guiding expert routing to suppress error propagation from unreliable signals. This approach is the first to enable continuous, fine-grained modeling of modality quality, allowing a single model to generalize across diverse degradation conditions. It achieves state-of-the-art performance on multiple benchmarks and demonstrates practical utility through a “one checkpoint fits all scenarios” capability.

Abstract

Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, multimodal inputs are often compromised by dynamic noise or modality missingness. Existing methods typically treat these imperfections as discrete cases or assume fixed corruption ratios, which limits their adaptability to continuously varying reliability conditions. To address this, we first introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework. Building on this, we propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty. This mechanism explicitly guides expert routing, enabling the model to suppress error propagation from unreliable signals while preserving task-relevant information. Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-All property in practice.

Problem

• Multimodal Sentiment Analysis
• modality missingness
• dynamic noise
• reliability spectrum
• quality degradation

Innovation

• Continuous Reliability Spectrum
• Quality-Aware Mixture of Experts
• Aleatoric Uncertainty
• Multimodal Sentiment Analysis
• Expert Routing