🤖 AI Summary
This study investigates whether large language models can substantively adapt their outputs in response to neurodiversity (ND)-related context provided in system prompts, distinguishing between superficial and structural adjustments. To this end, we introduce NDBench, a benchmark comprising 576 samples that integrates systematic prompt manipulation, adversarial masking, multidimensional human evaluation, and Krippendorff’s alpha reliability analysis. We propose the first measurement framework for differentiating levels of adaptation and find that merely stating an ND identity is insufficient to suppress harmful outputs. In contrast, explicit instructions significantly increase response length and structural coherence (p < 10⁻⁸) and reduce harmfulness by 36–44% along masking-augmented dimensions. The NDBench resource package is released publicly alongside this work.
📝 Abstract
We examine if frontier chat-based large language models (LLMs) adjust their outputs based on neurodivergence (ND) context in system prompts and describe the nature of these adjustments. Specifically, we propose NDBench, a 576-output benchmark involving two frontier models, three system prompt types (baseline, ND-profile assertion, and ND-profile assertion with explicit instructions for adjustments), four canonical ND profiles, and 24 prompts across four categories, one of which involves an adversarial masking strategy.
Four trends emerge consistently from our findings. First, LLMs show significant adaptation under ND context, where fully instructed conditions yield lengthier and more structured outputs, characterized by higher token counts, more headings, and more granular steps (p < 10^-8, Holm-corrected). Second, such adaptation is largely structural in nature: although list density does not change much, there is a marked rise in the frequency of headings and per-step detail. Third, ND persona assertion alone fails to suppress potentially harmful tendencies, as masking-reinforcement decreases only in explicitly instructed cases (36-44% reduction); the reduction rate barely changes in persona assertion conditions.
Moreover, reliability analysis of LLM-based harm assessment reveals that only two out of the six dimensions (masking and reinforcement, validation quality) exceed the pre-defined inter-judge agreement criterion (alpha >= 0.67) and thus can be considered primary results.
NDBench is made publicly available along with its prompts, outputs, code, and other resources, forming a reproducible framework for auditing future LLMs' adaptation to ND awareness.