Problem
Research questions and friction points this paper is trying to address.
multilinguality
embedding space
language coverage
There is No Theoretical Curse of Multilinguality For Embedding Space Structure
🤖 AI Summary
本文探讨了多语言模型性能下降问题，通过理论证明完美多语言性所需维度仅对数增长，表明无理论上的多语言诅咒，并以实证研究支持这一观点。
📝 Abstract
A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model. The curse of multilinguality describes the phenomenon of degradation in multilingual model performance as we increase language coverage, posing a threat to the above goal. This paper asks whether multilingual embedding spaces are inherently incapable of achieving perfect multilinguality without a prohibitive increase in required capacity. We first formalize the goal of "perfect multilinguality", embodied in two multilinguality conditions. We then prove that the minimum dimensionality required for perfect multilinguality grows only logarithmically in the number of languages. That is, we show that there is no theoretical curse of multilinguality for embedding space structure. This suggests that the empirical curse of multilinguality is a result of real world data and training conditions. We back this understanding with a small-scale empirical study. Our paper provides the first theoretical and intrinsic perspective on the curse of multilinguality, with implications for the scientific understanding of this phenomenon.
Innovation
Methods, ideas, or system contributions that make the work stand out.
perfect multilinguality
minimum dimensionality
logarithmic growth
curse of multilinguality
embedding space structure
🔎 Similar Papers
No similar papers found.
