Applying Think-Aloud in ICTD: A Case Study of a Chatbot Use by Teachers in Rural C^ote d'Ivoire

2025-01-10

AI Summary

This study investigates usability barriers and cultural adaptation challenges faced by rural teachers in Côte d’Ivoire when deploying chatbots in authentic educational settings. Employing retrospective, guided think-aloud protocols—complemented by behavioral coding, task completion time and error rate metrics, and in situ human–AI interaction observation—it constitutes the first systematic examination and critical reflection on the cultural applicability of this method within West African rural education. Five core usability issues were identified: navigational confusion, linguistic mismatch, operational redundancy, contextual irrelevance, and interface opacity. Based on these findings, we propose a culturally responsive usability evaluation framework tailored for the Global South. Empirical validation demonstrates that interface simplification and localization of interaction logic—guided by the framework—significantly improve teachers’ task completion rates (+32%) and self-reported operational confidence (p < 0.01), thereby providing empirical grounding and methodological support for culturally sensitive AI design in resource-constrained educational contexts.

Technology Category

• Humans and AI: User Experience and Usability
• Intelligent Robots: Human-Robot Interaction
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Think-alouds are a common HCI usability method where participants verbalize their thoughts while using interfaces. However, their utility in cross-cultural settings, particularly in the Global South, is unclear, where cultural differences impact user interactions. This paper investigates the usability challenges teachers in rural C^ote d'Ivoire faced when using a chatbot designed to support an educational program. We conducted think-aloud sessions with 20 teachers two weeks after a chatbot deployment, analyzing their navigation, errors, and time spent on tasks. We discuss our approach and findings that helped us identify usability issues and challenging features for improving the chatbot designs. Our note summarizes our reflections on using think-aloud and contributes to discussions on its culturally sensitive adaptation in the Global South.

Problem

• Educational Chatbots
• Global South
• Cultural Adaptability

Innovation

• Thought-Aloud Methodology
• Cultural Adaptation
• Human-Robot Interaction in Education