How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Study

2026-04-25

AI Summary

This study addresses a critical challenge in responsible AI (RAI): how large language model–driven tools obscure the attributability, transparency, and credibility of scholarly judgment during early-stage research. Through think-aloud protocols and in situ observation of 15 researchers engaged in literature exploration, synthesis, and research ideation, this work provides the first empirical account of AI’s impact on academic judgment in authentic research settings. Findings reveal that AI-generated outputs often convey unwarranted confidence that masks underlying cognitive uncertainty, while opaque generation and retrieval processes impair traceability. Moreover, trust in these systems proves highly fragile. In response, researchers deploy diverse compensatory strategies to reclaim epistemic agency, thereby situating AI-assisted scholarship within RAI principles and offering an empirical foundation for enhancing accountability and transparency in scientific practice.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Reasoning under Uncertainty: Causality
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Responsible Web: Algorithmic accountability and transparency on the web
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

In the early stages of scientific research, researchers rely on core scholarly judgments to identify relevant literature, assess credible evidence, and determine which directions merit pursuit. As AI tools become increasingly integrated into these early-stage workflows, the scholarly judgments that were once transparent and attributable to individual researchers become obscured, raising critical Responsible AI (RAI) concerns around accountability, transparency, and trust. Yet how these three dimensions manifest in real-time, in-situ scholarly practice remains largely unexplored. To address this gap, we conducted a think-aloud study with 15 researchers to examine how they used AI tools powered by large language models (LLMs) across early-stage research tasks, including literature exploration, synthesis, and research ideation. Our key findings address the tripartite constructs of accountability, transparency, and trust. First, the confident tone of AI outputs misrepresents epistemic uncertainty, making it more difficult for researchers (who are ultimately accountable) to identify which outputs require the greatest scrutiny. Second, opaque retrieval and content construction make provenance difficult to establish for transparency. Third, trust in AI is fragile, context-dependent, and easily eroded. In response, participant researchers were seen to develop compensatory strategies to restore scholarly judgment under uncertainty. Overall, our findings serve to contextualize AI-mediated research as a RAI problem grounded in lived researcher experience and motivate attention to deliberate AI integration that preserves accountability, supports transparency, and fosters informed trust.

Problem

• accountability
• transparency
• trust
• Responsible AI
• early-stage research

Innovation

• Responsible AI
• think-aloud study
• large language models
• scholarly judgment
• epistemic uncertainty