Instructions shape Production of Language, not Processing

2026-05-11

AI Summary

This study investigates how instructions modulate the internal language processing and generation mechanisms of large language models. Employing a cognitively inspired two-stage framework—integrating hierarchical probing, attention interventions, and diverse prompt variants—the work systematically analyzes the dynamics of task-relevant information in input and output tokens across five binary judgment tasks and multiple model families. The research reveals, for the first time, that instructions predominantly influence model behavior through the generation stage rather than the processing stage, with this asymmetry markedly amplified by increased model scale and instruction tuning. Task-related information in output tokens exhibits strong behavioral correlation and is significantly shaped by instructions, whereas such information in input tokens remains stable yet weakly associated with behavior. Causal validity of this mechanism is further confirmed through attention ablation experiments.

Abstract

Instructions trigger a production-centered mechanism in language models. Through a cognitively inspired lens that separates language processing and production, we reveal this mechanism as an asymmetry between the two stages by probing task-specific information layer-wise across five binary judgment tasks. Specifically, we measure how instruction tokens shape information both when sample tokens, the input under evaluation, are processed and when output tokens are produced. Across prompting variations, task-specific information in sample tokens remains largely stable and correlates only weakly with behavior, whereas the same information in output tokens varies substantially and correlates strongly with behavior. Attention-based interventions confirm this pattern causally: blocking instruction flow to all subsequent tokens reduces both behavior and information in output tokens, whereas blocking it only to sample tokens has minimal effect on either. The asymmetry generalizes across model families and tasks, and becomes sharper with model scale and instruction-tuning, both of which disproportionately affect the production stage. Our findings suggest that understanding model capabilities requires jointly assessing internals and behavior, while decomposing the internal perspective by token position to distinguish the processing of input tokens from the production of output tokens.

Problem

• instruction
• language processing
• language production
• asymmetry
• token representation

Innovation

• instruction tuning
• language production
• language processing
• layer-wise probing
• attention intervention