Offloading Score: Measuring AI Reliance Through Counterfactual Workflows

2026-05-28

AI Summary

Current measures of AI reliance primarily rely on output adoption or subjective self-reports, which inadequately capture the allocation of cognitive effort between users and AI during task execution. This work proposes a counterfactual workflow-based simulation method that models the steps users would take without AI assistance to quantify the proportion of cognitive effort offloaded to the AI. Introducing a novel metric—the Offloading Score—it provides a more precise measure of AI dependence. This score effectively captures dynamic shifts in reliance under time pressure, facilitating both user self-reflection and system-level interventions. In a programming study with 40 developers, the Offloading Score detected a statistically significant 43% increase in reliance under time pressure (p = 0.018), outperforming conventional metrics and revealing that heightened dependence manifests as increased delegation of subtasks and direct reuse of AI-generated outputs.

Abstract

AI tools are increasingly integrated into real-world workflows. However, existing measures of reliance on these tools focus on AI output adoption or on self-reported indicators, rather than how task effort is distributed between users and tools. Here, we introduce offloading score, a measure of reliance that quantifies the fraction of cognitive effort offloaded to an AI tool. Offloading Score is simulation-based -- we construct a counterfactual workflow by estimating how the user would have completed the task without the tool, and then computing the fraction of steps saved by using the tool. We validate offloading score through intrinsic evaluations of metric validity, and a controlled user study ($n=40$) with developers performing programming tasks using AI tools. We vary time pressure to test whether reliance measures capture the known increase in reliance under time pressure. We show that offloading score detects significantly higher reliance in time-constrained settings ($+43\%$, $p=0.018$), while usage-based and self-reported baseline measures of reliance do not distinguish the conditions. We complement this with descriptive insights showing that higher reliance manifests as greater delegation of subtasks to the tool and more direct reuse of AI outputs. Finally, we demonstrate an approach of using offloading score in combination with target outcomes of a task (e.g., code understanding) to identify when reliance may be (in)appropriate. Our framework offers two contributions: an instrument users can apply to measure and reflect on their own reliance, and a quantitative signal that agent designers can utilize to mitigate overreliance.

Problem

• AI reliance
• cognitive effort
• offloading score
• counterfactual workflow
• human-AI collaboration

Innovation

• offloading score
• counterfactual workflow
• AI reliance
• cognitive offloading
• human-AI collaboration