Stop using Media Bias/Fact Check in research

2026-07-13

AI Summary

Although Media Bias/Fact Check (MBFC) is widely employed in misinformation research, its methodology lacks academic rigor, relying instead on the subjective judgments of a single entity that are erroneously treated as objective and authoritative. This study systematically exposes MBFC’s nature as a “hegemonic computational representation” through a literature review, methodological critique, and discourse analysis, highlighting how it disguises political positioning as factual assessment. The findings reveal that existing scholarship largely overlooks MBFC’s subjectivity and methodological flaws, rendering it unsuitable as a foundational data source for academic inquiry. Challenging the scholarly community’s uncritical reliance on MBFC, this work calls for its discontinuation in research contexts and advocates for more critical scrutiny of media evaluation datasets.

Abstract

Media Bias/Fact Check (MBFC) purports to quantify the bias, credibility, and factuality of reporting for roughly 10,000 media sources, and the resulting data is commonly used in misinformation research. In the present study, we show that MBFC's methodology does not meet basic standards of rigor for academic research. Despite its widespread prevalence, studies using MBFC rarely examine it carefully, often describing it in ways that contradict its ``About'' page, or treating it as authoritative despite MBFC's disclaimer that it is ``not a tested scientific method... [but] a simple guide to the idea of a source's bias.'' We identified no papers that adequately describe MBFC as the opinions of a single person or critically engage with its methodology in order to justify proceeding with its use. We argue that MBFC's data is not neutral or accurate, but a computationally legible account of hegemony, a specious dataset for uncritical research that mistakes the familiarity of the concepts it quantifies with accuracy. Our study concludes with a call for academic researchers to stop using MBFC. MBFC's data quantifies the results of political processes, including campaigns to discredit the press, and presents them as simple facts about the world, thus reproducing the crisis misinformation scholarship exists to address.

Problem

• Media Bias/Fact Check
• misinformation research
• methodological rigor
• data bias
• academic credibility

Innovation

• Media Bias/Fact Check
• misinformation research
• methodological critique
• hegemony
• computational legibility