'Layer su Layer': Identifying and Disambiguating the Italian NPN Construction in BERT's family

2026-04-04

AI Summary

This study investigates how pretrained language models represent Italian noun-preposition-noun (NPN) constructions, challenging prevailing assumptions in construction grammar and probing methodologies. By deploying probe classifiers across BERT’s layers and incorporating construction family annotations, the research systematically analyzes the hierarchical encoding of both formal and semantic properties of NPN constructions within contextualized embeddings. Extending construction grammar to the low-resource language Italian, the work critically reexamines the theoretical foundations of probing experiments and establishes the first empirical bridge between constructionist linguistic theory and neural language models. The findings reveal that BERT encodes constructional information in a layered, hierarchical manner, offering novel insights into the interaction between linguistic structure and deep learning representations.

Abstract

Interpretability research has highlighted the importance of evaluating Pretrained Language Models (PLMs) and in particular contextual embeddings against explicit linguistic theories to determine what linguistic information they encode. This study focuses on the Italian NPN (noun-preposition-noun) constructional family, challenging some of the theoretical and methodological assumptions underlying previous experimental designs and extending this type of research to a lesser-investigated language. Contextual vector representations are extracted from BERT and used as input to layer-wise probing classifiers, systematically evaluating information encoded across the model's internal layers. The results shed light on the extent to which constructional form and meaning are reflected in contextual embeddings, contributing empirical evidence to the dialogue between constructionist theory and neural language modelling

Problem

• NPN construction
• Italian
• BERT
• constructional meaning
• contextual embeddings

Innovation

• layer-wise probing
• contextual embeddings
• NPN construction
• BERT interpretability
• constructionist theory