🤖 AI Summary
This work investigates the emergence mechanism of syntactic attention structures (SAS) during masked language modeling (MLM) pretraining and their causal role in grammatical competence development. We track Transformer-based MLM training trajectories end-to-end, combining attention pattern analysis, loss curvature estimation, and phase segmentation to precisely localize a phase-transition-like emergence of SAS within sharp loss-drop intervals. Through carefully designed causal intervention experiments—introducing targeted SAS suppression—we establish, for the first time, that SAS is a necessary intermediate state for grammatical capability emergence. Moreover, we find that transient SAS suppression improves final model performance, revealing competitive dynamics between SAS and other beneficial features. This study establishes the first measurable and intervenable paradigm for syntactic evolution, providing empirically grounded, syntax-structured explanations and theoretical foundations for stage-wise capability emergence in large language models.
📝 Abstract
Most interpretability research in NLP focuses on understanding the behavior and features of a fully trained model. However, certain insights into model behavior may only be accessible by observing the trajectory of the training process. We present a case study of syntax acquisition in masked language models (MLMs) that demonstrates how analyzing the evolution of interpretable artifacts throughout training deepens our understanding of emergent behavior. In particular, we study Syntactic Attention Structure (SAS), a naturally emerging property of MLMs wherein specific Transformer heads tend to focus on specific syntactic relations. We identify a brief window in pretraining when models abruptly acquire SAS, concurrent with a steep drop in loss. This breakthrough precipitates the subsequent acquisition of linguistic capabilities. We then examine the causal role of SAS by manipulating SAS during training, and demonstrate that SAS is necessary for the development of grammatical capabilities. We further find that SAS competes with other beneficial traits during training, and that briefly suppressing SAS improves model quality. These findings offer an interpretation of a real-world example of both simplicity bias and breakthrough training dynamics.
