AV-TLX for measuring (mental) workload while driving AVs: Born from NASA-TLX but developed for the era of automated vehicles

2025-03-26

AI Summary

Conventional mental workload scales (e.g., NASA-TLX, DALI) exhibit limited validity for Level 3 (L3) automated driving due to novel human–machine interaction demands, particularly during conditional automation and take-over scenarios. Method: We developed AV-TLX—the first domain-specific mental workload assessment tool for L3 automation—through narrative literature review and iterative qualitative workshops. Its novel 8-dimensional structure uniquely decouples workload associated with “take-over tasks” from that of “autonomous driving tasks,” enabling granular, dimension-wise analysis. Scale validation employed content validity indices (CVR > 0.78; CVI met thresholds) and dynamic high-fidelity driving simulation. Contribution/Results: The final 19-item instrument comprises eight subscales, achieves Cronbach’s α = 0.86, and correlates moderately with NASA-TLX (r = 0.6). Fully open-sourced, AV-TLX establishes a standardized, interpretable benchmark for human factors evaluation in L3 automated driving.

Technology Category

• Application Domains: Mobility, Driving & Flight
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The introduction of automated vehicles has redefined the level of interaction between the driver and the vehicle, introducing new tasks and so impose different workloads. Existing tools such as NASA-TLX and DALI are still used to assess driving workload in automated vehicles, despite not accounting for new tasks. This study introduces AV-TLX, a specialized tool for measuring workload in Level 3 automated driving. The development process began with a narrative literature review to identify the primary factors influencing workload. This was followed by a series of qualitative sessions during which the dimensions and later the questions of the questionnaire were designed. The tools validity was first assessed using CVR and CVI indices, and its reliability and convergent validity were evaluated using a dynamic driving simulator with high fidelity. The final version of AV-TLX comprises 19 questions across 8 subscales, demonstrating excellent reliability (0.86) and validity (CVR>0.78). An agreement scores between the results of AV-TLX and NASA-TLX in the simulation study was 0.6, which is considered acceptable for the consistency of two questionnaires. Furthermore, this questionnaire can be utilized in two ways. First by reporting the overall workload and/or divided into 8 primary subscales, or by categorizing the questions into two groups including takeover task workload and automated driving task workload. The final version of this questionnaire, as presented in the paper, is available for use in future studies focusing on Level 3 automated driving.

Problem

• Develop AV-TLX to measure workload in Level 3 automated driving
• Address limitations of NASA-TLX and DALI for automated vehicles
• Validate AV-TLX reliability and validity for workload assessment

Innovation

• Developed AV-TLX for Level 3 automated driving
• Validated using CVR, CVI, and driving simulator
• Includes 19 questions across 8 subscales