Problem
Research questions and friction points this paper is trying to address.
multimodal emotion recognition
reliability-aware
dialogue-aware
ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation
🤖 AI Summary
为解决对话中多模态情感识别对不同证据源可靠性的适应问题，提出ReH-FUSE框架，通过文本、音频及跨模态专家的决策级路由实现自适应融合。
📝 Abstract
Multimodal emotion recognition in conversation (ERC) requires adapting to the instance-dependent reliability of different evidence sources. Lexical content may be decisive, vocal expression may provide complementary cues, or accurate recognition may require cross-modal interaction; fixed fusion does not explicitly account for this variation. We propose ReH-FUSE, a reliability-aware framework with dialogue-aware text, audio, and cross-modal experts. Its decision-level router first models the relative preference between text and audio and then balances the resulting unimodal mixture against the cross-modal expert. This factorization separates unimodal competition from cross-modal selection. Across three independent runs on IEMOCAP, ReH-FUSE achieves 74.34% weighted F1 and 73.11% macro F1; on MELD, it achieves 68.03% weighted F1. Controlled ablations show that learned routing outperforms uniform expert averaging and benefits from cross-modal interaction.
Innovation
Methods, ideas, or system contributions that make the work stand out.
reliability-aware
hierarchical fusion
multimodal emotion recognition
dialogue-aware
🔎 Similar Papers
No similar papers found.
