Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures

2026-03-18

AI Summary

This study addresses the systematic gender bias in decoder-only machine translation models, which arises from mismatches in gender marking between languages—particularly when mapping implicit gender cues in the source language to explicit gendered forms in the target language. The authors introduce a fine-grained gender disambiguation diagnostic framework and propose a novel metric, “prior bias,” to quantify a model’s default gender inclination. This metric is applied for the first time to evaluate decoder-only architectures. Experimental results show that such models do not consistently outperform traditional encoder-decoder models on gender-related translation tasks; however, post-training strategies like instruction tuning significantly enhance their contextual awareness and effectively reduce male prior bias.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

While Large Language Models achieve state-of-the-art results across a wide range of NLP tasks, they remain prone to systematic biases. Among these, gender bias is particularly salient in MT, due to systematic differences across languages in whether and how gender is marked. As a result, translation often requires disambiguating implicit source signals into explicit gender-marked forms. In this context, standard benchmarks may capture broad disparities but fail to reflect the full complexity of gender bias in modern MT. In this paper, we extend recent frameworks on bias evaluation by: (i) introducing a novel measure coined "Prior Bias", capturing a model's default gender assumptions, and (ii) applying the framework to decoder-only MT models. Our results show that, despite their scale and state-of-the-art status, decoder-only models do not generally outperform encoder-decoder architectures on gender-specific metrics; however, post-training (e.g., instruction tuning) not only improves contextual awareness but also reduces the masculine Prior Bias.

Problem

• gender disambiguation
• machine translation
• gender bias
• decoder-only architectures
• Prior Bias

Innovation

• Prior Bias
• gender disambiguation
• decoder-only models
• bias evaluation
• instruction tuning