From Measurement to Mitigation: Exploring the Transferability of Debiasing Approaches to Gender Bias in Maltese Language Models

2025-07-03

AI Summary

This study systematically investigates gender bias in low-resource, morphologically rich Maltese language models (BERTu/mBERTu), focusing on the cross-lingual transferability of English-centric debiasing methods—including counterfactual data augmentation, Auto-Debias, GuiDebias, and dropout regularization. Leveraging cross-lingual benchmarks (CrowS-Pairs, SEAT), we construct the first Maltese-specific gender bias evaluation dataset. Experiments reveal that state-of-the-art debiasing techniques underperform significantly on Maltese compared to English, highlighting their sensitivity to morphological complexity and training-data scarcity. Our contributions are threefold: (1) empirical validation of the limitations of prevailing debiasing paradigms in low-resource multilingual settings; (2) release of a reusable evaluation dataset and an open experimental framework for Maltese and similar languages; and (3) a call to action—and foundational evidence—for developing morphology-aware, language-specific bias mitigation strategies.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The advancement of Large Language Models (LLMs) has transformed Natural Language Processing (NLP), enabling performance across diverse tasks with little task-specific training. However, LLMs remain susceptible to social biases, particularly reflecting harmful stereotypes from training data, which can disproportionately affect marginalised communities. We measure gender bias in Maltese LMs, arguing that such bias is harmful as it reinforces societal stereotypes and fails to account for gender diversity, which is especially problematic in gendered, low-resource languages. While bias evaluation and mitigation efforts have progressed for English-centric models, research on low-resourced and morphologically rich languages remains limited. This research investigates the transferability of debiasing methods to Maltese language models, focusing on BERTu and mBERTu, BERT-based monolingual and multilingual models respectively. Bias measurement and mitigation techniques from English are adapted to Maltese, using benchmarks such as CrowS-Pairs and SEAT, alongside debiasing methods Counterfactual Data Augmentation, Dropout Regularization, Auto-Debias, and GuiDebias. We also contribute to future work in the study of gender bias in Maltese by creating evaluation datasets. Our findings highlight the challenges of applying existing bias mitigation methods to linguistically complex languages, underscoring the need for more inclusive approaches in the development of multilingual NLP.

Problem

• Measure gender bias in Maltese language models
• Explore transferability of debiasing methods to Maltese
• Address bias challenges in low-resource, gendered languages

Innovation

• Transfer debiasing methods to Maltese language models
• Adapt English bias benchmarks to Maltese
• Create evaluation datasets for Maltese gender bias