AIMA at SemEval-2024 Task 10: History-Based Emotion Recognition in Hindi-English Code-Mixed Conversations

2025-01-19

International Workshop on Semantic Evaluation

AI Summary

This work addresses emotion recognition in Hindi-English code-mixed (Hinglish) dialogues. We propose a history-aware multimodal framework comprising two key components: (1) a Hinglish-to-English translation pre-processing pipeline for linguistic normalization, and (2) a joint contextual modeling architecture integrating bidirectional LSTM or Transformer-based context encoders with an ensemble of multilingual pretrained language models (BERT, RoBERTa, XLM-R). Crucially, we introduce the novel concept of “history-aware contextual modeling”, synergistically coupled with code-mixed translation pre-processing. This design enhances cross-lingual robustness—particularly critical in low-resource emotion recognition in conversations (ERC). Evaluated on SemEval-2024 Task 10 Subtask 1, our approach outperforms all baseline systems, demonstrating the efficacy of jointly leveraging contextual awareness and language normalization for code-mixed emotion classification.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In this study, we introduce a solution to the SemEval 2024 Task 10 on subtask 1, dedicated to Emotion Recognition in Conversation (ERC) in code-mixed Hindi-English conversations. ERC in code-mixed conversations presents unique challenges, as existing models are typically trained on monolingual datasets and may not perform well on code-mixed data. To address this, we propose a series of models that incorporate both the previous and future context of the current utterance, as well as the sequential information of the conversation. To facilitate the processing of code-mixed data, we developed a Hinglish-to-English translation pipeline to translate the code-mixed conversations into English. We designed four different base models, each utilizing powerful pre-trained encoders to extract features from the input but with varying architectures. By ensembling all of these models, we developed a final model that outperforms all other baselines.

Problem

• Sentiment Analysis
• Code-Switching
• Hinglish Chat

Innovation

• Multimodal Framework
• Contextual Sequential Information
• Advanced Pre-trained Models