Ethics and Social Responsibility in AI-Assisted Interviewing: An LLM-in-the-Loop Study of AI-Generated Follow-Up Questions

2026-06-29

AI Summary

This study addresses the limited depth of follow-up questioning in semi-structured interviews, which often stems from interviewers’ high cognitive load and constrained domain knowledge. Through a Wizard-of-Oz experiment integrating GPT-4o in a human-in-the-loop setting, the research enables human interview日晚间 to selectively adopt and edit AI-generated probes during real-time conversations, preserving human oversight while exploring viable models for AI-assisted interviewing. The work systematically identifies five interwoven ethical risks inherent in such AI-augmented interactions: harmful language, attentional distraction, unequal participation, blurred accountability, and privacy compliance concerns. Building on these findings, the study proposes design and governance recommendations centered on safety, respect, and accountability, offering an empirical foundation and guiding principles for the development of responsible AI interview systems.

Abstract

Semi-structured interviews rely on timely, context-sensitive follow-up questions, yet interviewers' cognitive load and limited domain familiarity can constrain probing depth. We report findings from an LLM-in-the-loop Wizard-of-Oz (WoZ) study that simulates an AI follow-up assistant in live interviewing while preserving human oversight. In our setup, a co-interviewer selectively relayed and could edit AI-generated follow-up questions (AGQs) produced in real time by GPT-4o, enabling a realistic approximation of deployment without fully automating the interaction. Across 17 interviewers with varied qualitative-method expertise, participants raised five interlocking concerns: (1) harmful or discriminatory language and unpredictable interaction harms, (2) undermining interviewees' sense of respect through divided attention and missing nonverbal cues, (3) technology-based participation inequality, (4) unclear responsibility when harms occur, and (5) privacy, disclosure, and compliance risks when AI listens, records, or transcribes sensitive content. We translate these concerns into design and governance implications for safer, more respectful, and more accountable AI-assisted interviewing.

Problem

• AI ethics
• social responsibility
• AI-assisted interviewing
• follow-up questions
• privacy risks

Innovation

• LLM-in-the-loop
• AI-assisted interviewing
• follow-up question generation
• Wizard-of-Oz study
• human-AI collaboration