Taxonomy of Risks on Automated Fact-Checking Systems Considering its Propagation

2026-06-24

AI Summary

This study addresses the safety and societal risks posed by automated fact-checking systems, which may inadvertently propagate misinformation or defamatory content due to erroneous judgments. Recognizing that existing IT security frameworks such as STRIDE inadequately capture the unique risks inherent to these systems, this work proposes the first fine-grained, three-stage risk propagation model—linking risk factors, hazardous scenarios, and resulting harms—augmented with guided keywords for systematic qualitative analysis. The approach identifies 32 concrete risks and has been successfully applied to the DEFAME system, uncovering novel vulnerabilities overlooked by conventional methods. By offering a structured methodology tailored to high-stakes AI applications, this research establishes a new paradigm for evaluating and mitigating risks in automated fact-checking and other safety-critical artificial intelligence systems.

Abstract

In recent years, the posting of fake news including disinformation and misinformation on social networking services (SNS) has become a social problem. To combat this fake news, fact-checking that is the process of assessing the veracity of posts on SNS has become increasingly important. While fact-checking is currently performed by fact-checking organizations, it is difficult to fact-check all posts on SNS. Therefore, the use of automated fact-checking systems is effective. Recent automated fact-checking systems utilize artificial intelligence and large language models, so there are risks of incorrect judgments and posting incorrect results on social media which can lead to the spread of misinformation or to engage in defamation. In this paper, as a first step toward enabling the safe use of automated fact-checking systems, we categorize the specific risks on automated fact-checking systems. In this categorizing, we consider a three-stage risk propagation: risk factors, hazardous situations, and harm. Our analysis revealed that 32 specific risks exist in automated fact-checking systems. In this paper, we utilize the categorized risks as analytical cues (guide words) to present the risk assessment of the automated fact-checking system DEFAME. This assessment result indicates that risks that cannot be derived using STRIDE, a conventional IT security risk assessment method can be derived using our guide words.

Problem

• automated fact-checking
• risk taxonomy
• misinformation
• AI safety
• risk propagation

Innovation

• risk taxonomy
• automated fact-checking
• risk propagation
• large language models
• safety assessment