MEGA: xLSTM with Multihead Exponential Gated Fusion for Precise Aspect-based Sentiment Analysis

2025-07-01

AI Summary

Existing ABSA methods struggle to balance computational efficiency with modeling capacity: deep learning models lack global contextual awareness, Transformers incur prohibitive computational overhead, and Mamba-style approaches suffer from CUDA dependency and weakened local modeling. To address these limitations, we propose MEGA—a multi-head exponential gating fusion framework built upon xLSTM. MEGA innovatively introduces a partially reversed backward mLSTM stream to enhance local perception and designs a Multi-Head Cross-exponential Gating and Fusion (MECGAF) mechanism to dynamically integrate bidirectional long- and short-term semantics. This architecture achieves linear-time complexity while jointly strengthening short-range dependency modeling and global context capture. Evaluated on three standard ABSA benchmarks, MEGA consistently outperforms state-of-the-art methods, achieving superior accuracy and inference efficiency simultaneously.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Mixture of Experts (MoE)
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Aspect-based Sentiment Analysis (ABSA) is a critical Natural Language Processing (NLP) task that extracts aspects from text and determines their associated sentiments, enabling fine-grained analysis of user opinions. Existing ABSA methods struggle to balance computational efficiency with high performance: deep learning models often lack global context, transformers demand significant computational resources, and Mamba-based approaches face CUDA dependency and diminished local correlations. Recent advancements in Extended Long Short-Term Memory (xLSTM) models, particularly their efficient modeling of long-range dependencies, have significantly advanced the NLP community. However, their potential in ABSA remains untapped. To this end, we propose xLSTM with Multihead Exponential Gated Fusion (MEGA), a novel framework integrating a bi-directional mLSTM architecture with forward and partially flipped backward (PF-mLSTM) streams. The PF-mLSTM enhances localized context modeling by processing the initial sequence segment in reverse with dedicated parameters, preserving critical short-range patterns. We further introduce an mLSTM-based multihead cross exponential gated fusion mechanism (MECGAF) that dynamically combines forward mLSTM outputs as query and key with PF-mLSTM outputs as value, optimizing short-range dependency capture while maintaining global context and efficiency. Experimental results on three benchmark datasets demonstrate that MEGA outperforms state-of-the-art baselines, achieving superior accuracy and efficiency in ABSA tasks.

Problem

• Balancing computational efficiency and high performance in ABSA
• Enhancing local and global context modeling in sentiment analysis
• Overcoming CUDA dependency and local correlation issues in Mamba

Innovation

• Bi-directional mLSTM with PF-mLSTM for context
• Multihead cross exponential gated fusion mechanism
• Combines forward and backward streams dynamically