Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns

2026-07-13

AI Summary

This study investigates whether general-purpose multimodal language models can simulate the systematic naming errors exhibited by individuals with post-stroke aphasia. By applying controlled noise perturbations to different layers of the LLaVA 1.6 model, the authors successfully reproduce seven distinct error patterns observed in a cohort of 278 patients performing a picture-naming task. A neural classifier is employed to annotate and align these simulated errors with clinical data. The work presents the first high-fidelity reconstruction of individual patients’ multi-category error profiles using a generic model not specifically designed for clinical applications, introducing a novel “digital twin for aphasia” paradigm. Experimental results show that at least six error types were accurately replicated for 97.8% of patients, with 79.5% achieving full matching across all seven categories, and the simulated error distributions closely mirroring those from clinical observations.

Abstract

Aphasia following stroke commonly produces systematic naming errors with characteristic profiles, but whether general-purpose language models not designed for clinical simulation can reproduce these patterns remains untested. We investigated (1) whether lesions or controlled perturbations to a multimodal language model can reproduce different types of errors in picture naming, and (2) whether the framework can reproduce the complete error profile of individual persons with aphasia (PWAs). Using LLaVA 1.6, we evaluated perturbation configurations that varied the layer, proportion, and amount of noise applied to model units. We examined 278 PWAs on the Philadelphia Naming Test, classifying responses into seven categories using a validated neural classifier. Six of seven response categories (correct, semantic, mixed, unrelated, neologism, no response errors) emerged at clinically-comparable proportions across distinct parameter space regions, with formal paraphasia being the exception. Searching the perturbation space revealed configurations that reproduced the individual error profile in at least six of seven categories for 97.8% of PWAs and in all seven categories for 79.5% of PWAs. Monte Carlo baselines confirmed that this matching reflects joint inter-category structure rather than marginal overlap. These results establish a quantitative framework for reproducing individual aphasic error patterns in picture naming. They suggest the potential for language models to serve as digital twins of individuals with post-stroke aphasia.

Problem

• aphasia
• picture naming
• language models
• naming errors
• stroke

Innovation

• lesioned language models
• aphasia simulation
• multimodal perturbation
• digital twin
• picture naming errors