Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification

2026-04-12

AI Summary

This work addresses the challenges of modeling long-range semantic dependencies and accurately interpreting ambiguous sentiment expressions in lengthy movie reviews. To this end, the authors propose a lightweight hybrid architecture that, for the first time, integrates a dynamic adaptive multi-head attention mechanism with supervised contrastive learning within a BERT encoder. The dynamic attention mechanism adaptively adjusts head weights to emphasize sentiment-critical information, while supervised contrastive learning enhances intra-class compactness and inter-class separability in the embedding space. Evaluated on the IMDB dataset, the proposed method achieves an accuracy of 94.67%, outperforming strong baseline models by 1.5–2.5 percentage points. The approach maintains computational efficiency and scalability while significantly improving robustness to noise and generalization capability.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The exponential growth of user-generated movie reviews on digital platforms has made accurate text sentiment classification a cornerstone task in natural language processing. Traditional models, including standard BERT and recurrent architectures, frequently struggle to capture long-distance semantic dependencies and resolve ambiguous emotional expressions in lengthy review texts. This paper proposes a novel hybrid framework that seamlessly integrates dynamic adaptive multi-head attention with supervised contrastive learning into a BERT-based Transformer encoder. The dynamic adaptive attention module employs a global context pooling vector to dynamically regulate the contribution of each attention head, thereby focusing on critical sentiment-bearing tokens while suppressing noise. Simultaneously, the supervised contrastive learning branch enforces tighter intra-class compactness and larger inter-class separation in the embedding space. Extensive experiments on the IMDB dataset demonstrate that the proposed model achieves competitive performance with an accuracy of 94.67\%, outperforming strong baselines by 1.5--2.5 percentage points. The framework is lightweight, efficient, and readily extensible to other text classification tasks.

Problem

• text sentiment classification
• long-distance semantic dependencies
• ambiguous emotional expressions
• user-generated reviews

Innovation

• Dynamic Adaptive Attention
• Supervised Contrastive Learning
• BERT-based Transformer
• Sentiment Classification
• Global Context Pooling