The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models

2026-06-11

AI Summary

This study investigates how language models balance between storing multiword units holistically and applying abstract compositional rules, with a focus on whether Verb+up phrasal constructions form independent representations. Employing representational probing techniques, it presents the first systematic comparison of how large language models (LLMs) and automatic speech recognition (ASR) models internally encode Verb+up phrases, analyzing activation patterns in relation to frequency and predictability metrics. The findings reveal that both model types exhibit a tendency to store Verb+up constructions holistically when they are high-frequency yet low-predictability, offering cross-modal empirical support for usage-based theories of linguistic representation and underscoring the universality of experience-driven mechanisms in language processing.

Abstract

A crucial aspect of linguistic capability is the ability to trade off between stored representations and abstract knowledge: one must retrieve learned representations, but also generate novel ones by applying productive rules. While recent work has examined abstract knowledge in language models, holistic storage of multi-word units has received far less attention. We probe internal representations in text-based LLMs and an ASR model, testing whether V+up phrasal verbs develop distinct representations as a function of frequency and predictability. All models show evidence of holistic storage driven by frequency and predictability, further supporting usage-based theories of language.

Problem

• holistic storage
• phrasal verbs
• language models
• frequency
• predictability

Innovation

• holistic storage
• phrasal verbs
• frequency effects
• predictability
• usage-based theory