SRWToolkit: An Open Source Wizard of Oz Toolkit to Create Social Robotic Avatars

2025-09-04

AI Summary

Current social robot avatar development tools rely on cloud-based large language models (LLMs), lacking modularity, local execution, and offline capability—thereby hindering scalable, reproducible research in character customization and human-robot interaction. This paper introduces a web-based prototype development tool powered by locally deployed LLMs, supporting multimodal real-time configuration of appearance, behavior, language, and voice, along with text/speech input and wake-word activation. By integrating local inference, real-time text-to-speech (TTS), keyword spotting, and a Wizard-of-Oz collaborative framework, the tool ensures privacy preservation, offline operation, and high extensibility. A small-scale user study (n=11) validates its usability, user experience, and perceived credibility across diverse roles—including hospital receptionist and mathematics tutor—demonstrating significant improvements in agility and reproducibility for social robotics research.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Natural Language Processing: Safety and Robustness
• Humans and AI: AI for Accessibility

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present SRWToolkit, an open-source Wizard of Oz toolkit designed to facilitate the rapid prototyping of social robotic avatars powered by local large language models (LLMs). Our web-based toolkit enables multimodal interaction through text input, button-activated speech, and wake-word command. The toolkit offers real-time configuration of avatar appearance, behavior, language, and voice via an intuitive control panel. In contrast to prior works that rely on cloud-based LLM services, SRWToolkit emphasizes modularity and ensures on-device functionality through local LLM inference. In our small-scale user study ($n=11$), participants created and interacted with diverse robotic roles (hospital receptionist, mathematics teacher, and driving assistant), which demonstrated positive outcomes in the toolkit's usability, trust, and user experience. The toolkit enables rapid and efficient development of robot characters customized to researchers' needs, supporting scalable research in human-robot interaction.

Problem

• Facilitates rapid prototyping of social robotic avatars
• Enables multimodal interaction through local LLMs
• Supports customizable avatar appearance and behavior configuration

Innovation

• Open-source Wizard of Oz toolkit
• Local LLM inference for on-device functionality
• Real-time multimodal interaction configuration