How Do Teachers Create Pedagogical Chatbots?: Current Practices and Challenges

2025-03-02

AI Summary

This study addresses the practical challenges and integration pathways involved in K–12 teachers’ autonomous development of pedagogical chatbots. Drawing on semi-structured interviews with seven in-service teachers and rigorous qualitative coding, it systematically uncovers a three-tiered authoring practice—encompassing instructional task design, technical implementation, and dialogue evaluation—and identifies three prototypical authoring patterns alongside five recurrent barriers. Notably, the analysis reveals significant disparities in capability bottlenecks between novice and experienced teachers. Building on these findings, the study proposes a low-threshold, teacher-centered authoring support framework grounded in empirical evidence. This framework advances the design of educationally effective AI tools by prioritizing pedagogical agency and technical accessibility. As the first empirical investigation focused explicitly on teacher-led chatbot creation, the work fills a critical gap in the literature on educator-centered educational AI and contributes substantively to the pragmatic integration of AI-augmented instruction in real-world classrooms.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

AI chatbots have emerged as promising educational tools for personalized learning experiences, with advances in large language models (LLMs) enabling teachers to create and customize these chatbots for their specific classroom needs. However, there is a limited understanding of how teachers create pedagogical chatbots and integrate them into their lessons. Through semi-structured interviews with seven K-12 teachers, we examined their practices and challenges when designing, implementing, and deploying chatbots. Our findings revealed that teachers prioritize developing task-specific chatbots aligned with their lessons. Teachers engaged in various creation practices and had different challenges; novices struggled mainly with initial design and technical implementation, while experienced teachers faced challenges with technical aspects and analyzing conversational data. Based on these insights, we explore approaches to supporting teachers' chatbot creation process and opportunities for designing future chatbot creation systems. This work provides practical insights into deepening the understanding of teacher-driven AI chatbots and AI-augmented learning environments.

Problem

• Understanding teacher practices in creating pedagogical chatbots.
• Identifying challenges in chatbot design and implementation.
• Exploring support systems for teacher-driven AI chatbot creation.

Innovation

• Teachers create task-specific pedagogical chatbots.
• Semi-structured interviews reveal creation practices.
• Future systems aim to support chatbot design.