Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs

2025-10-29

AI Summary

Cross-lingual alignment (CLA) facilitates multilingual knowledge transfer but often incurs “cultural erasure”—a loss of culturally specific response capabilities. This work identifies an inherent tension between knowledge transfer and cultural preservation. To address it, we propose *surgical guidance*: a hierarchical activation control mechanism that decouples factual knowledge transfer from cultural localization across model layers. Leveraging a newly constructed dual-dimension evaluation framework—spanning transfer fidelity and cultural localization—we integrate internal representation analysis with inference-time layered intervention. Empirical validation across six languages demonstrates that our method significantly improves cultural contextualization quality while fully preserving cross-lingual knowledge transfer performance—overcoming the fundamental trade-off inherent in existing CLA approaches.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Cross-lingual alignment (CLA) aims to align multilingual representations, enabling Large Language Models (LLMs) to seamlessly transfer knowledge across languages. While intuitive, we hypothesize, this pursuit of representational convergence can inadvertently cause "cultural erasure", the functional loss of providing culturally-situated responses that should diverge based on the query language. In this work, we systematically analyze this trade-off by introducing a holistic evaluation framework, the transfer-localization plane, which quantifies both desirable knowledge transfer and undesirable cultural erasure. Using this framework, we re-evaluate recent CLA approaches and find that they consistently improve factual transfer at the direct cost of cultural localization across all six languages studied. Our investigation into the internal representations of these models reveals a key insight: universal factual transfer and culturally-specific knowledge are optimally steerable at different model layers. Based on this finding, we propose Surgical Steering, a novel inference-time method that disentangles these two objectives. By applying targeted activation steering to distinct layers, our approach achieves a better balance between the two competing dimensions, effectively overcoming the limitations of current alignment techniques.

Problem

• Balancing knowledge transfer and cultural preservation in multilingual models
• Quantifying trade-off between factual transfer and cultural erasure
• Disentangling universal and culture-specific knowledge across model layers

Innovation

• Surgical Steering method disentangles transfer and cultural objectives
• Targeted activation steering applied to distinct model layers
• Balances factual transfer with cultural localization in multilingual LLMs