Knowing Is Not Enough: Information Retrievability as a Precondition to Effective LLM Oversight

2026-09-01

AI Summary

研究通过自我生成解释和提示激活方法，解决大语言模型在组织工作中因信息难以检索导致的人类监督失效问题。

Abstract

Large language models (LLMs) are increasingly embedded in organizational work, yet their errors often pass human review. Prior research locates such failures in users' capability to review LLM output or their engagement in doing so. We develop an alternative, retrieval-based account of human oversight and posit that error detection is more effective when oversight-relevant information is accessible to users at the moment of review. Across two randomized lab-in-the-field experiments with 640 customer-facing employees, we show that self-generated explanations improve error detection and strengthen recall of verification-relevant reasoning, while cues that reactivate such reasoning help sustain detection under repeated LLM use. Theoretically, we identify information retrievability as a distinct precondition for effective oversight and specify generative encoding and cue-supported reactivation as mechanisms that build and sustain it. Practically, lightweight onboarding self-explanations and daily retrieval cues can make human oversight more resilient as LLM use becomes routine.

Problem

• Large language models
• human oversight
• information retrievability
• error detection

Innovation

• Information Retrievability
• Oversight
• Generative Encoding
• Cue-Supported Reactivation