LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety

2026-04-13

AI Summary

This work addresses the significant degradation in safety performance of large language models (LLMs) when applied to low-resource languages, which stems from a misalignment between their semantic understanding capabilities and safety mechanisms predominantly trained on high-resource languages. The study identifies, for the first time, a language-agnostic semantic bottleneck layer within the model’s internal representations and directly anchors safety alignment in this semantic space, thereby overcoming the limitations of conventional surface-level textual alignment. By integrating semantic bottleneck analysis, language-invariant alignment, and geometric modeling of internal representations, the proposed approach substantially enhances cross-lingual safety: it reduces attack success rates on LLaMA-3.1-8B-Instruct from 24.7% to 2.8% and consistently maintains rates between 3–4% across the Qwen2.5 and Qwen3 model families (7B–32B).

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried in low-resource languages. We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages. Consistent with this hypothesis, we empirically identify the semantic bottleneck in LLMs, an intermediate layer in which the geometry of model representations is governed primarily by shared semantic content rather than language identity. Building on this observation, we propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks. Experiments show that LASA substantially improves safety across all languages: average attack success rate (ASR) drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains around 3-4% across Qwen2.5 and Qwen3 Instruct models (7B-32B). Together, our analysis and method offer a representation-level perspective on LLM safety, suggesting that safety alignment requires anchoring safety understanding not in surface text, but in the model's language-agnostic semantic space.

Problem

• LLM safety
• low-resource languages
• semantic alignment
• language-agnostic
• safety vulnerability

Innovation

• semantic bottleneck
• language-agnostic alignment
• LLM safety
• representation geometry
• cross-lingual safety