IOHunter: Graph Foundation Model to Uncover Online Information Operations

2024-12-19

arXiv.org

AI Summary

Information operations (IO) orchestrated by foreign actors on social media pose severe threats to democratic integrity and societal trust. However, existing detection methods suffer from insufficient labeled data and poor generalization across events and national contexts. To address these challenges, we propose the first graph-based foundation model paradigm for IO detection, integrating large language models (for semantic understanding) with graph neural networks (to model heterogeneous user–content–propagation relationships). Our end-to-end multi-source graph representation learning framework unifies supervised, weakly supervised, and cross-domain detection scenarios. Evaluated on real-world IO datasets spanning six countries, our method achieves state-of-the-art performance, with an average improvement of 23.6% across key metrics. It demonstrates significantly enhanced robustness, cross-context transferability, and practical deployability—advancing both theoretical foundations and operational utility in IO defense.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Social media platforms have become vital spaces for public discourse, serving as modern agor'as where a wide range of voices influence societal narratives. However, their open nature also makes them vulnerable to exploitation by malicious actors, including state-sponsored entities, who can conduct information operations (IOs) to manipulate public opinion. The spread of misinformation, false news, and misleading claims threatens democratic processes and societal cohesion, making it crucial to develop methods for the timely detection of inauthentic activity to protect the integrity of online discourse. In this work, we introduce a methodology designed to identify users orchestrating information operations, a.k.a. extit{IO drivers}, across various influence campaigns. Our framework, named exttt{IOHunter}, leverages the combined strengths of Language Models and Graph Neural Networks to improve generalization in emph{supervised}, emph{scarcely-supervised}, and emph{cross-IO} contexts. Our approach achieves state-of-the-art performance across multiple sets of IOs originating from six countries, significantly surpassing existing approaches. This research marks a step toward developing Graph Foundation Models specifically tailored for the task of IO detection on social media platforms.

Problem

• Detect malicious actors orchestrating information operations on social media.
• Improve generalization in detecting IO drivers using AI models.
• Protect online discourse integrity by identifying inauthentic activities.

Innovation

• Combines Language Models and Graph Neural Networks
• Detects information operations across multiple campaigns
• Achieves state-of-the-art performance in IO detection