RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning based on Emotional Information

2024-06-16

arXiv.org

AI Summary

To address the limitations of fine-tuning dependency, model complexity, and neglect of affective signals in cross-domain fake news detection, this paper proposes Affect-RAG—the first retrieval-augmented large language model framework integrating affective information. Affect-RAG embeds affective features throughout the entire pipeline: retrieval, indexing, and few-shot in-context learning (ICL), constructing an affect-aware unified vector index to enable fine-tuning-free cross-domain generalization. Its key contributions are threefold: (1) the first systematic integration of affect into both RAG and ICL paradigms; (2) the first fine-tuning-free approach achieving robust cross-domain fake news detection; and (3) substantial performance gains—up to 31.18% absolute improvement over state-of-the-art few-shot methods across three benchmark datasets—while significantly reducing computational overhead and annotation costs.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Misinformation is prevalent in various fields such as education, politics, health, etc., causing significant harm to society. However, current methods for cross-domain misinformation detection rely on effort- and resource-intensive fine-tuning and complex model structures. With the outstanding performance of LLMs, many studies have employed them for misinformation detection. Unfortunately, they focus on in-domain tasks and do not incorporate significant sentiment and emotion features (which we jointly call {em affect}). In this paper, we propose RAEmoLLM, the first retrieval augmented (RAG) LLMs framework to address cross-domain misinformation detection using in-context learning based on affective information. RAEmoLLM includes three modules. (1) In the index construction module, we apply an emotional LLM to obtain affective embeddings from all domains to construct a retrieval database. (2) The retrieval module uses the database to recommend top K examples (text-label pairs) from source domain data for target domain contents. (3) These examples are adopted as few-shot demonstrations for the inference module to process the target domain content. The RAEmoLLM can effectively enhance the general performance of LLMs in cross-domain misinformation detection tasks through affect-based retrieval, without fine-tuning. We evaluate our framework on three misinformation benchmarks. Results show that RAEmoLLM achieves significant improvements compared to the other few-shot methods on three datasets, with the highest increases of 15.64%, 31.18%, and 15.73% respectively. This project is available at https://github.com/lzw108/RAEmoLLM.

Problem

• Detects cross-domain misinformation using emotional context
• Overcomes reliance on fine-tuning with retrieval-augmented LLMs
• Improves detection accuracy via affective information integration

Innovation

• Uses emotional LLM for affective embeddings
• Retrieves top K examples via RAG
• Applies in-context learning without fine-tuning