Probing Experts' Perspectives on AI-Assisted Public Speaking Training

2025-07-10

AI Summary

This study investigates the pedagogical effectiveness and design limitations of AI-assisted tools for business public speaking training. Method: We conducted a qualitative study with 16 public speaking experts via semi-structured interviews and focus groups, systematically analyzing their evaluations of existing AI systems and eliciting design recommendations. Contribution/Results: We propose an “expert-feedback-driven” framework for AI tool optimization, whose core innovations include explainable, personalized real-time feedback and a principled re-allocation of roles between AI and human coaches—where AI handles technical assessments (e.g., prosody, pausing, vocal delivery) and coaches focus on higher-order rhetorical strategies and affective engagement. Empirical findings confirm that hybrid training models integrating AI capabilities with evidence-based instructional design significantly enhance training efficiency and user acceptance. The study provides empirical grounding and actionable design principles for human-AI collaborative intelligent education.

Technology Category

• Humans and AI: User Experience and Usability
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Background: Public speaking is a vital professional skill, yet it remains a source of significant anxiety for many individuals. Traditional training relies heavily on expert coaching, but recent advances in AI has led to novel types of commercial automated public speaking feedback tools. However, most research has focused on prototypes rather than commercial applications, and little is known about how public speaking experts perceive these tools. Objectives: This study aims to evaluate expert opinions on the efficacy and design of commercial AI-based public speaking training tools and to propose guidelines for their improvement. Methods: The research involved 16 semi-structured interviews and 2 focus groups with public speaking experts. Participants discussed their views on current commercial tools, their potential integration into traditional coaching, and suggestions for enhancing these systems. Results and Conclusions: Experts acknowledged the value of AI tools in handling repetitive, technical aspects of training, allowing coaches to focus on higher-level skills. However they found key issues in current tools, emphasising the need for personalised, understandable, carefully selected feedback and clear instructional design. Overall, they supported a hybrid model combining traditional coaching with AI-supported exercises.

Problem

• Evaluating expert opinions on AI public speaking tools
• Assessing efficacy and design of commercial AI training tools
• Proposing guidelines for improving AI-assisted speech coaching

Innovation

• AI tools handle repetitive technical training aspects
• Hybrid model combines coaching with AI exercises
• Personalised feedback enhances public speaking training