🤖 AI Summary
This study addresses the growing challenge of misinformation amplified by social media and AI technologies, where traditional manual fact-checking proves inadequate. The authors propose a novel multi-agent system that uniquely integrates consensus mechanisms from human annotations, cognitive and knowledge diversity, and hierarchical collaborative structures. Built upon open-source large language models—including LLaMA, Qwen, Kimi, Deepseek, and LLaMA-Nemotron—the framework enables automated detection and verification of false claims. Evaluated on English, Polish, Slovak, and Bulgarian datasets, the approach significantly outperforms monolithic models such as GPT-4 and GPT-3.5 across three key tasks: direct misinformation identification, filtering of claims requiring verification, and detection of verifiable factual statements. The system also demonstrates high transparency and reproducibility.
📝 Abstract
In contemporary societies, the threat of disinformation has reached alarming levels, exacerbated by the proliferation of electronic communication, social media, and advancements in artificial intelligence. As a result, there is an urgent need to develop effective countermeasures to mitigate this menace. However, the sheer scale of the problem renders manual fact-checking and human-based verification inadequate, underscoring the necessity for automated methods to detect and debunk disinformation. This article proposes a novel approach based on a multi-agent system that emulates the decision-making processes of human annotators engaged in disinformation detection tasks. By incorporating a consensus mechanism, diversity in cognition and diversity in knowledge, and also hierarchical structure, inspired by human annotators' behavior, the proposed method achieves superior results compared to individual Large Language Models (LLMs), including GPT 4 and GPT 3.5. The system leverages open models (e.g., LLaMA, Kimi, Qwen, Deepseek and LLaMA-Nemotron) to ensure greater transparency. The evaluation of the proposed method encompasses datasets in languages with varying resource availability, including English (high-resource), Polish (medium-resource), Slovak (low-resource) and Bulgarian (low-resource). Experiments were conducted on tasks such as direct disinformation detection, identification of texts worthy of verification, and detection of texts containing verifiable factual claims.