G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment

2026-06-17

AI Summary

Cross-lingual idiom alignment is challenging due to the non-compositional nature of idioms and their weak correspondence with surface forms, a problem exacerbated in low-resource languages where reliance on literal translation fails. This work proposes G-IdiomAlign, the first benchmark that leverages English glosses as an explicit semantic pivot to construct a high-confidence idiom alignment dataset. It introduces two evaluation protocols: multiple-choice equivalence judgment with type-based distractors and gloss-based contrastive generation. Analyses using Wiktionary glosses, embedding similarity metrics, and large language models (e.g., Qwen3-8B) reveal a pervasive bias toward literal translations—particularly pronounced in low-resource settings. Incorporating glosses significantly improves semantic alignment quality, yet overall performance remains suboptimal, with effectiveness largely attributable to variations at the attention head level.

Abstract

Idioms are difficult to transfer across languages due to their non-compositionality and weak surface-form grounding, making literal mappings unreliable. We present G-IdiomAlign, a gloss-pivoted benchmark where each idiom is anchored by an English gloss from Wiktionary. We further construct a high-confidence reference alignment set for reproducible evaluation. G-IdiomAlign supports two protocols: (1) a controlled Multiple-Choice Idiom Equivalence with typed distractors for error attribution; and (2) a Gloss-Contrastive Generation contrasting No-gloss and With-gloss inputs to isolate the effect of an explicit semantic pivot. Across diverse LLMs, a bias to literal translation is a dominant failure mode, especially when the target is a low-resource language. Glosses consistently improve Gloss-Contrastive Generation under an embedding-based semantic proxy, but performance remains modest, indicating substantial headroom in the open output space. Subsequent analysis on Qwen3-8B further suggests that cross-condition differences are concentrated more in attention heads than in layers, while better With-gloss generations coincide with stronger gloss anchoring.

Problem

• cross-lingual idiom alignment
• non-compositionality
• literal translation
• semantic pivot
• idiom equivalence

Innovation

• gloss-pivoted alignment
• cross-lingual idiom
• semantic anchoring
• contrastive generation
• idiom non-compositionality