Behavioral Indicators of Overreliance During Interaction with Conversational Language Models

2026-02-12

AI Summary

This study addresses the challenge of identifying excessive reliance on conversational large language models (LLMs) in human-AI interaction—a subtle issue often undetectable post-hoc—by proposing a real-time assessment method based on behavioral logs. The authors deployed LLMs injected with controlled hallucinations to 77 participants across three realistic tasks, quantifying reliance through users’ ability to detect and correct errors. Combining semantic encoding with clustering analysis of interaction logs, they systematically identified five distinct behavioral patterns significantly associated with overreliance, including frequent copy-pasting, skipping initial comprehension, and repetitive LLM referencing. These patterns reveal fundamental differences between high- and low-reliance users in task understanding, navigation strategies, and information processing, offering empirical foundations for designing effective intervention mechanisms.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Interaction Techniques and Devices

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

LLMs are now embedded in a wide range of everyday scenarios. However, their inherent hallucinations risk hiding misinformation in fluent responses, raising concerns about overreliance on AI. Detecting overreliance is challenging, as it often arises in complex, dynamic contexts and cannot be easily captured by post-hoc task outcomes. In this work, we aim to investigate how users'behavioral patterns correlate with overreliance. We collected interaction logs from 77 participants working with an LLM injected plausible misinformation across three real-world tasks and we assessed overreliance by whether participants detected and corrected these errors. By semantically encoding and clustering segments of user interactions, we identified five behavioral patterns linked to overreliance: users with low overreliance show careful task comprehension and fine-grained navigation; users with high overreliance show frequent copy-paste, skipping initial comprehension, repeated LLM references, coarse locating, and accepting misinformation despite hesitation. We discuss design implications for mitigation.

Problem

• overreliance
• conversational language models
• behavioral indicators
• misinformation
• human-AI interaction

Innovation

• behavioral patterns
• overreliance detection
• conversational LLMs
• interaction logs
• misinformation acceptance