Misinformation Span Detection in Videos via Audio Transcripts

2026-04-23

AI Summary

This study addresses a critical limitation in existing video misinformation detection methods, which typically assess the veracity of entire videos without identifying the precise locations of false content. To enable fine-grained localization, this work introduces the novel task of video misinformation span detection and constructs two new publicly available datasets comprising over 500 videos with more than 2,400 annotated false claim spans, accompanied by synchronized audio, video, and transcript data. Leveraging advanced language models integrated with speech transcription and temporal alignment techniques, the proposed approach achieves an F1 score of 0.68 on this task, substantially enhancing both the interpretability and precision of video-based misinformation detection.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Responsible Web: Mitigating misinformation and disinformation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Online misinformation is one of the most challenging issues lately, yielding severe consequences, including political polarization, attacks on democracy, and public health risks. Misinformation manifests in any platform with a large user base, including online social networks and messaging apps. It permeates all media and content forms, including images, text, audio, and video. Distinctly, video-based misinformation represents a multifaceted challenge for fact-checkers, given the ease with which individuals can record and upload videos on various video-sharing platforms. Previous research efforts investigated detecting video-based misinformation, focusing on whether a video shares misinformation or not on a video level. While this approach is useful, it only provides a limited and non-easily interpretable view of the problem given that it does not provide an additional context of when misinformation occurs within videos and what content (i.e., claims) are responsible for the video's misinformation nature. In this work, we attempt to bridge this research gap by creating two novel datasets that allow us to explore misinformation detection on videos via audio transcripts, focusing on identifying the span of videos that are responsible for the video's misinformation claim (misinformation span detection). We present two new datasets for this task. We transcribe each video's audio to text, identifying the video segment in which the misinformation claims appears, resulting in two datasets of more than 500 videos with over 2,400 segments containing annotated fact-checked claims. Then, we employ classifiers built with state-of-the-art language models, and our results show that we can identify in which part of a video there is misinformation with an F1 score of 0.68. We make publicly available our annotated datasets. We also release all transcripts, audio and videos.

Problem

• misinformation span detection
• video misinformation
• audio transcripts
• fact-checked claims
• fine-grained detection

Innovation

• misinformation span detection
• video-based misinformation
• audio transcript analysis
• fine-grained fact-checking
• annotated video dataset