AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation

2024-09-15

arXiv.org

AI Summary

Children with autism—particularly those with minimally verbal profiles—face bidirectional communication barriers with caregivers. Method: We propose a tablet-based AI-assisted system featuring a novel “dual-path real-time collaborative guidance” mechanism: (1) dynamic, evidence-informed interaction strategy prompts for caregivers, and (2) context-aware, situation-adapted visual vocabulary cards for children. The system integrates a lightweight multimodal situational understanding model, real-time interaction state tracking, personalized card generation algorithms, and human–AI co-designed interfaces. Contribution/Results: Our work foregrounds communicative trade-offs and child agency, moving beyond unidirectional intervention paradigms. In a two-week in-the-wild deployment across 11 families, we observed significant increases in dialogue frequency and turn-taking instances; enhanced caregiver reflective capacity regarding interaction strategies; improved child-initiated expression; and 100% engagement across all participants.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

As minimally verbal autistic (MVA) children communicate with parents through few words and nonverbal cues, parents often struggle to encourage their children to express subtle emotions and needs and to grasp their nuanced signals. We present AACessTalk, a tablet-based, AI-mediated communication system that facilitates meaningful exchanges between an MVA child and a parent. AACessTalk provides real-time guides to the parent to engage the child in conversation and, in turn, recommends contextual vocabulary cards to the child. Through a two-week deployment study with 11 MVA child-parent dyads, we examine how AACessTalk fosters everyday conversation practice and mutual engagement. Our findings show high engagement from all dyads, leading to increased frequency of conversation and turn-taking. AACessTalk also encouraged parents to explore their own interaction strategies and empowered the children to have more agency in communication. We discuss the implications of designing technologies for balanced communication dynamics in parent-MVA child interaction.

Problem

• Autism
• Communication Impairment
• Parenting Strategies

Innovation

• AACessTalk
• Artificial Intelligence
• Autism Communication