Investigating the Effects of LLM Use on Critical Thinking Under Time Constraints: Access Timing and Time Availability

2026-03-09

AI Summary

This study investigates how the timing of large language model (LLM) access and time constraints jointly influence human critical thinking performance. In a between-subjects experiment (N = 393), participants completed a real-world, document-driven decision-making task under manipulated conditions of LLM availability (early, continuous, late, or none) and time pressure (sufficient vs. insufficient). The findings reveal a significant interaction between these factors, including a novel “time reversal effect”: early LLM access enhances performance under time pressure but impairs critical thinking when time is abundant. These results demonstrate that temporal context critically determines whether LLM assistance augments or undermines cognitive performance, offering essential theoretical and practical implications for the design of LLM-augmented systems and the evaluation of human–AI collaboration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The impact of large language models (LLMs) on critical thinking has provoked growing attention, yet this impact on actual performance may not be uniformly negative or positive. Particularly, the role of time -- the temporal context under which an LLM is provided -- remains overlooked. In a between-subjects experiment (n=393), we examined two types of time constraints for a critical thinking task requiring participants to make a reasoned decision for a real-world scenario based on diverse documents: (1) LLM access timing -- an LLM available only at the beginning (early), throughout (continuous), near the end (late), or not at all (no LLM), and (2) time availability -- insufficient or sufficient time for the task. We found a temporal reversal: LLM access from the start (early, continuous) improved performance under time pressure but impaired it with sufficient time, whereas beginning the task independently (late, no LLM) showed the opposite pattern. These findings demonstrate that time constraints fundamentally shape whether an LLM augments or undermines critical thinking, making time a central consideration when designing LLM support and evaluating human-AI collaboration in cognitive tasks.

Problem

• large language models
• critical thinking
• time constraints
• human-AI collaboration
• cognitive tasks

Innovation

• large language models
• critical thinking
• time constraints
• human-AI collaboration
• access timing