Subjective Multi-Bias Detection with Large Language Models

2026-08-10

AI Summary

This work addresses the challenge of subjective bias in text that undermines authenticity and reliability by proposing a unified large language model–based approach for joint detection of multiple bias types. Leveraging over 4,000 sentence pairs from the WIKIBIAS corpus—derived from Wikipedia edits—the method enables, for the first time, fine-grained, multi-span identification of framing bias, epistemic bias, and demographic bias within a single framework, while also producing corresponding classification labels. The proposed approach substantially improves automatic detection accuracy across all three categories of subjective bias. To foster reproducible research on bias, the authors have publicly released their implementation code.

Abstract

In this project, we delved into the pervasive challenge of bias detection within the text content. More specifically, our focus lies on the identification of subjective bias, a type of bias that introduces improper attitudes or portrays a statement at odds with the actual truth. The subjective bias can jeopardize the authenticity and reliability of texts, leading to misconceptions and potential social tensions, especially when expressed through offensive language. Following prior work [1], we tackled with three different types of subjective biases in text: (1) framing bias with the use of one-sided words or phrases containing a particular point of view; (2) epistemological bias which includes subtle linguistic features that can affect the believability of the texts; (3) demographic bias with word/phrase usage under presuppositions of a particular demographic factor (i.e., gender or religion). In terms of the data we utilize, the input consists of texts that may harbor subjective biases. The output is a classification or annotation that reveals the presence or absence of such biases within the provided content. More specifically, we detected three different types of multi-span biases in corpus WIKIBIAS [2] with more than 4,000 sentence pairs from Wikipedia edits. The data is labelled by bias type for span pairs with the following categories: (1) framing bias, (2) epistemological bias, (3) demographic bias, and (4) no bias. The project codes are released at https://github.com/HoningJade/LLM-Bias-Type-Classification.

Problem

• subjective bias
• framing bias
• epistemological bias
• demographic bias
• bias detection

Innovation

• subjective bias detection
• large language models
• multi-span classification
• framing bias
• epistemological bias