Designing Conversational AI to Support Think-Aloud Practice in Technical Interview Preparation for CS Students

2025-07-18

AI Summary

Computer science students lack structured think-aloud training for technical interviews, and the role and user perception of conversational AI in this context remain poorly understood. Method: We designed a large language model–based conversational AI system integrating three novel components: (1) interaction mechanisms enhancing social presence; (2) multimodal, multidimensional feedback—spanning logical reasoning, verbal expression, and problem-solving strategy—beyond textual output; and (3) a human-AI collaborative, crowdsourced demonstration-learning paradigm. Contribution/Results: Through user interaction experiments and in-depth qualitative analysis, we found high user acceptance of AI in both simulated interviews and demonstration-based learning. The study distills four evidence-based design principles for effective practice and identifies critical socio-technical challenges—including fairness, trust calibration, and feedback personalization. Our work provides a reusable methodology and empirical foundation for leveraging AI to scaffold higher-order thinking skills in technical interview preparation.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

One challenge in technical interviews is the think-aloud process, where candidates verbalize their thought processes while solving coding tasks. Despite its importance, opportunities for structured practice remain limited. Conversational AI offers potential assistance, but limited research explores user perceptions of its role in think-aloud practice. To address this gap, we conducted a study with 17 participants using an LLM-based technical interview practice tool. Participants valued AI's role in simulation, feedback, and learning from generated examples. Key design recommendations include promoting social presence in conversational AI for technical interview simulation, providing feedback beyond verbal content analysis, and enabling crowdsourced think-aloud examples through human-AI collaboration. Beyond feature design, we examined broader considerations, including intersectional challenges and potential strategies to address them, how AI-driven interview preparation could promote equitable learning in computing careers, and the need to rethink AI's role in interview practice by suggesting a research direction that integrates human-AI collaboration.

Problem

• Supporting think-aloud practice in technical interviews for CS students
• Exploring user perceptions of conversational AI in interview preparation
• Addressing limited structured practice opportunities with AI tools

Innovation

• LLM-based interview simulation tool
• Feedback beyond verbal content analysis
• Crowdsourced examples via human-AI collaboration