Measuring Large Language Models Dependency: Validating the Arabic Version of the LLM-D12 Scale

2025-08-23

AI Summary

A critical gap exists in culturally adapted, validated instruments for assessing psychological dependency on large language models (LLMs) among Arabic-speaking populations. Method: This study developed and validated the Arabic version of the LLM-D12 scale through a rigorous cross-cultural adaptation process, employing confirmatory factor analysis, Cronbach’s α reliability testing (overall α = 0.90), and criterion-related validity analyses—demonstrating significant associations with AI acceptance, internet addiction, and need for cognition. Contribution/Results: The scale exhibits robust psychometric properties and cultural appropriateness, confirming its two-factor structure (instrumental and relational dependency). This is the first empirically validated LLM dependency instrument for Arabic-speaking contexts, addressing a key methodological void in AI-related psychological research in the region. It provides a reliable, culturally grounded measurement tool to support future cross-cultural comparative studies and targeted interventions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

There is an urgent need for reliable, culturally validated instruments to assess psychological responses to AI in general and large language models (LLMs). This need is global issue, but it is especially urgent among Arabic-speaking populations, where AI and LLMs adoption is accelerating, yet psychometric tools remain limited. This study presents the first validation of the LLM-D12, a dual-dimensional scale assessing Instrumental and Relationship Dependency on LLMs, in an Arab sample. A total of 250 Arab participants completed the Arabic version of the LLM-D12. Confirmatory Factor Analysis confirms the original 2-factor structure of LLM-D12 with all items showing good loading of corresponding Instrumental and Relationship Dependency. The scale showed good to excellent internal reliability (Cronbach alpha is 0.90 for Total, 0.85 for Instrumental Dependency, and 0.90 for Relationship Dependency). External validation revealed that Instrumental Dependency was positively associated with AI acceptance and internet addiction, while Relationship Dependency was linked to lower need for cognition and greater trustworthiness of LLM, demonstrating sensitivity of this instrument to different use and personal factors. These findings confirm that Arabic LLM-D12 is a psychometrically sound, culturally appropriate instrument, offering a necessary tool for research, education, and policy concerning AI and LLMs engagement in Arab contexts.

Problem

• Validating Arabic LLM-D12 scale for AI dependency assessment
• Measuring instrumental and relationship dependency on large language models
• Addressing psychometric tool gaps in Arabic-speaking populations

Innovation

• Validated Arabic LLM-D12 scale psychometrically
• Confirmed two-factor structure with CFA
• Assessed instrumental and relationship dependency dimensions