Problem
Research questions and friction points this paper is trying to address.
cross-linguistic syntactic regularities
processing mechanisms
multilingual LMs
Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models
🤖 AI Summary
本文使用因果干预方法研究多语言模型中的句法机制，发现跨语言机制转移与语言类型相似度有关，为跨语言句法结构提供新假设。
📝 Abstract
Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar mechanisms. However, this hypothesis has been difficult to test empirically due to our lack of fine-grained, manipulable access of human processing mechanisms. In this work, we take advantage of techniques from mechanistic interpretability to study such a question in multilingual LMs. We first isolate language-internal mechanisms before attempting to transfer them cross-lingually. Across four models and three well-studied constructions (subject--verb number agreement, anaphoric pronoun gender agreement, and filler--gap object extraction) we find consistent cross-lingual mechanism transfer. We further find transfer to be graded, with more transfer between more typologically similar languages. We believe our work provides novel hypotheses about cross-linguistic syntactic structures and multilingual processing, and more broadly shows how the study of language models can help inform linguistic theory.
Innovation
Methods, ideas, or system contributions that make the work stand out.
causal interventions
cross-lingual mechanism transfer
typological similarity
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
