MORA: AI-Mediated Story-Based practice for Speech Sound Disorder from Clinic to Home

2025-10-27

AI Summary

Preschool children with phonological disorders face challenges including low adherence to home-based practice, limited interactivity in existing story-based interventions, and poor alignment with clinical protocols. This paper proposes an AI-mediated interactive storytelling training system that bridges clinical and home settings: it embeds target phonemes within role-driven dynamic narratives, integrates phoneme-level speech recognition with real-time visual feedback, and enables remote configuration of content and therapy workflows by speech-language pathologists—establishing an AI-in-the-loop closed-loop management framework. The system employs a multimodal, child-friendly interface and a configurable workflow architecture. Expert evaluation confirms strong clinical alignment, significantly improved child engagement, and enhanced phoneme generalization, thereby validating the efficacy and feasibility of clinician–caregiver collaborative intervention.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Speech sound disorder is among the most common communication challenges in preschool children. Home-based practice is essential for effective therapy and for acquiring generalization of target sounds, yet sustaining engaging and consistent practice remains difficult. Existing story-based activities, despite their potential for sound generalization and educational benefits, are often underutilized due to limited interactivity. Moreover, many practice tools fail to sufficiently integrate speech--language pathologists into the process, resulting in weak alignment with clinical treatment plans. To address these limitations, we present MORA, an interactive story-based practice system. MORA introduces three key innovations. First, it embeds target sounds and vocabulary into dynamic, character-driven conversational narratives, requiring children to actively produce speech to progress the story, thereby creating natural opportunities for exposure, repetition, and generalization. Second, it provides visual cues, explicit instruction, and feedback, allowing children to practice effectively either independently or with caregivers. Third, it supports an AI-in-the-loop workflow, enabling SLPs to configure target materials, review logged speech with phoneme-level scoring, and adapt therapy plans asynchronously -- bridging the gap between clinic and home practice while respecting professional expertise. A formative study with six licensed SLPs informed the system's design rationale, and an expert review with seven SLPs demonstrated strong alignment with established articulation-based treatments, as well as potential to enhance children's engagement and literacy. Furthermore, discussions highlight the design considerations for professional support and configurability, adaptive and multimodal child interaction, while highlighting MORA's broader applicability across speech and language disorders.

Problem

• Addressing limited interactivity in speech sound disorder practice tools
• Bridging the gap between clinical treatment and home practice
• Enhancing child engagement through AI-mediated story-based activities

Innovation

• Embeds target sounds into interactive conversational narratives
• Provides visual cues and feedback for independent practice
• Enables AI-assisted therapist configuration and progress review