Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language

2025-07-22

AI Summary

This study addresses the challenge of assessing gender bias in large language models (LLMs) for German, revealing the risks of directly transferring English-centric bias measurement methods across languages. To this end, we construct five German-specific datasets grounded in linguistic gender theory—first systematically characterizing the ambiguity of male-associated occupational terms and the interference effect of grammatically neuter nouns on gender attribution. We further propose the first morphology- and syntax-aware, multi-dimensional bias evaluation framework tailored to German. Empirical evaluation across eight state-of-the-art multilingual LLMs demonstrates that gender bias patterns in German diverge significantly from those in English. Our work provides a reproducible German bias benchmark and open-source datasets, and advances multilingual bias assessment from “literal translation-based transfer” toward “language-aware, linguistically grounded customization.”

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

In recent years, various methods have been proposed to evaluate gender bias in large language models (LLMs). A key challenge lies in the transferability of bias measurement methods initially developed for the English language when applied to other languages. This work aims to contribute to this research strand by presenting five German datasets for gender bias evaluation in LLMs. The datasets are grounded in well-established concepts of gender bias and are accessible through multiple methodologies. Our findings, reported for eight multilingual LLM models, reveal unique challenges associated with gender bias in German, including the ambiguous interpretation of male occupational terms and the influence of seemingly neutral nouns on gender perception. This work contributes to the understanding of gender bias in LLMs across languages and underscores the necessity for tailored evaluation frameworks.

Problem

• Evaluating gender bias transferability from English to German in LLMs
• Developing German datasets for gender bias assessment in LLMs
• Addressing unique German gender bias challenges like ambiguous occupational terms

Innovation

• Developed five German gender bias datasets
• Evaluated eight multilingual LLM models
• Highlighted unique German gender bias challenges