"I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration

2026-05-20

AI Summary

Existing approaches struggle to dynamically attribute the respective contributions of humans and large language models (LLMs) to the formation and evolution of shared goals during collaboration. This work proposes CoTrace, a novel framework that introduces, for the first time, a goal-level attribution mechanism by decomposing explicit goals into verifiable requirements and distinguishing between the LLM’s direct contributions and indirect influences across dialogue turns. Through analysis of dialogue logs, controlled experiments, and user studies involving 638 real-world collaborative sessions, we find that model contributions account for 11–26% of goal development and significantly drive the introduction of specific requirements. Furthermore, visualizing attribution results reduces users’ cognitive bias regarding their reliance on AI by nearly two points on a five-point scale, highlighting the critical role of LLMs in the co-evolution of collaborative goals.

Abstract

As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.

Problem

• goal-level attribution
• human-AI collaboration
• contribution measurement
• large language models
• AI-assisted work

Innovation

• goal-level attribution
• human-AI collaboration
• CoTrace
• contribution tracing
• large language models