GKnow: Measuring the Entanglement of Gender Bias and Factual Gender

2026-05-12

AI Summary

Existing approaches struggle to disentangle factual gender-related knowledge from gender bias in language models, and there is a lack of unified benchmarks capable of evaluating both simultaneously. This work introduces GKnow, a benchmark designed to systematically assess a model’s ability to distinguish between these two types of information across diverse gender-related tasks. Combining neuron ablation and circuit analysis techniques, the study conducts experiments on GKnow, DiFair, and StereoSet. It reveals—for the first time—that factual gender knowledge and gender bias are highly entangled at both the neuronal and circuit levels. While neuron ablation can reduce bias, it concurrently degrades factual knowledge, a trade-off obscured by existing benchmarks, thereby highlighting critical limitations in current debiasing methods.

Abstract

Recent works have analyzed the impact of individual components of neural networks on gendered predictions, often with a focus on mitigating gender bias. However, mechanistic interpretations of gender tend to (i) focus on a very specific gender-related task, such as gendered pronoun prediction, or (ii) fail to distinguish between the production of factually gendered outputs (the correct assumption of gender given a word that carries gender as a semantic property) and gender biased outputs (based on a stereotype). To address these issues, we curate \gknow, a benchmark to assess gender knowledge and gender bias in language models across different types of gender-related predictions. \gknow allows us to identify and analyze circuits and individual neurons responsible for gendered predictions. We test the impact of neuron ablation on benchmarks for disentangling stereotypical and factual gender (DiFair and the test set of GKnow), as well as StereoSet. Results show that gender bias and factual gender are severely entangled on the level of both circuits and neurons, entailing that ablation is an unreliable debiasing method. Furthermore, we show that benchmarks for evaluating gender bias can hide the decrease in factual gender knowledge that accompanies neuron ablation. We curate GKnow as a contribution to the continuous development of robust gender bias benchmarks.

Problem

• gender bias
• factual gender
• language models
• stereotypes
• bias evaluation

Innovation

• gender bias
• factual gender
• neuron ablation
• circuit analysis
• bias benchmark