Dual Alignment Between Language Model Layers and Human Sentence Processing

2026-04-20

AI Summary

This study investigates how predictive capabilities across different layers of large language models (LLMs) correspond to variations in human cognitive effort during natural reading and syntactic ambiguity resolution. By regressing layer-specific surprisal and probability update measures from LLMs against human eye-tracking or reading time data, the authors identify and formally name a “dual alignment” phenomenon: natural reading primarily relies on shallow, weak predictive signals, whereas processing syntactically complex structures requires deep, context-sensitive representations. Building on this insight, they propose a multi-layer modeling approach that integrates probability updates from both shallow and deep layers. This method significantly outperforms single-layer surprisal models in predicting cognitive load, particularly mitigating the underestimation bias of deep representations in syntactic ambiguity contexts.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.

Problem

• sentence processing
• syntactic ambiguity
• cognitive effort
• language models
• surprisal

Innovation

• dual alignment
• syntactic ambiguity
• surprisal
• language model layers
• cognitive effort