Iterative Multilingual Spectral Attribute Erasure

2025-06-12

AI Summary

Existing debiasing methods are confined to monolingual modeling and thus struggle to identify and eliminate shared bias subspaces across multilingual representations. To address this, we propose the first iterative multilingual spectral attribute erasure framework, which jointly aligns, disentangles, and zero-shot erases cross-lingual bias subspaces via coupled spectral analysis and SVD-driven iterative subspace truncation. Our method requires no labeled data in target languages, transcending both monolingual and simplistic alignment paradigms. Evaluated across eight languages and five sensitive attributes, it significantly outperforms state-of-the-art approaches while preserving downstream task performance nearly intact on BERT, LLaMA, and Mistral models. This work establishes the first unsupervised, collaborative, and transferable bias suppression technique for multilingual representations—enabling effective, language-agnostic debiasing without compromising model utility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Multilingual representations embed words with similar meanings to share a common semantic space across languages, creating opportunities to transfer debiasing effects between languages. However, existing methods for debiasing are unable to exploit this opportunity because they operate on individual languages. We present Iterative Multilingual Spectral Attribute Erasure (IMSAE), which identifies and mitigates joint bias subspaces across multiple languages through iterative SVD-based truncation. Evaluating IMSAE across eight languages and five demographic dimensions, we demonstrate its effectiveness in both standard and zero-shot settings, where target language data is unavailable, but linguistically similar languages can be used for debiasing. Our comprehensive experiments across diverse language models (BERT, LLaMA, Mistral) show that IMSAE outperforms traditional monolingual and cross-lingual approaches while maintaining model utility.

Problem

• Debiasing multilingual word embeddings across languages
• Identifying joint bias subspaces in multiple languages
• Improving zero-shot debiasing without target language data

Innovation

• Iterative SVD-based truncation for debiasing
• Joint bias subspaces across multiple languages
• Zero-shot debiasing using linguistically similar languages