A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit

2025-02-13

AI Summary

To address fake news detection under information overload, this paper proposes a hybrid Transformer model enhanced by Bayesian-optimized BiGRU and integrated with TF-IDF text representations for end-to-end classification. The method combines the contextual modeling strength of Transformer with the sequential dependency capture capability of BiGRU, while leveraging TF-IDF for complementary lexical features. Crucially, this work introduces Bayesian hyperparameter optimization to the BiGRU-Transformer joint architecture for the first time—significantly improving tuning efficiency and generalization. Experimental results demonstrate rapid convergence within only 10 training epochs, achieving a test accuracy of 99.73% (a 0.06% improvement over baselines) and perfect 100% training accuracy. The model exhibits high precision, strong robustness against noisy inputs, and low computational overhead, making it suitable for real-time deployment. This approach establishes a novel paradigm for lightweight, efficient, and scalable fake news identification in high-volume digital environments.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Machine Learning: Hardware-aware ML

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

In this paper, we propose an optimized Transformer model that integrates Bayesian algorithms with a Bidirectional Gated Recurrent Unit (BiGRU), and apply it to fake news classification for the first time. First, we employ the TF-IDF method to extract features from news texts and transform them into numeric representations to facilitate subsequent machine learning tasks. Two sets of experiments are then conducted for fake news detection and classification: one using a Transformer model optimized only with BiGRU, and the other incorporating Bayesian algorithms into the BiGRU-based Transformer. Experimental results show that the BiGRU-optimized Transformer achieves 100% accuracy on the training set and 99.67% on the test set, while the addition of the Bayesian algorithm maintains 100% accuracy on the training set and slightly improves test-set accuracy to 99.73%. This indicates that the Bayesian algorithm boosts model accuracy by 0.06%, further enhancing the detection capability for fake news. Moreover, the proposed algorithm converges rapidly at around the 10th training epoch with accuracy nearing 100%, demonstrating both its effectiveness and its fast classification ability. Overall, the optimized Transformer model, enhanced by the Bayesian algorithm and BiGRU, exhibits excellent continuous learning and detection performance, offering a robust technical means to combat the spread of fake news in the current era of information overload.

Problem

• Fake news detection using hybrid Transformer model.
• Bayesian optimization enhances fake news classification accuracy.
• BiGRU-optimized Transformer achieves near-perfect detection performance.

Innovation

• Hybrid Transformer model
• Bayesian optimization integration
• BiGRU-enhanced fake news detection