Learning Japanese with Jouzu: Interaction Outcomes with Stylized Dialogue Fictional Agents

2025-07-08

AI Summary

This study investigates how stylized voice agents—specifically anime-inspired virtual characters—affect user interaction experience in multimodal Japanese language learning. Addressing learners with varying proficiency levels and cultural backgrounds, we design an asynchronous dialogue system integrating character persona, expressive text-to-speech (TTS), and stylistically diverse linguistic output; large language models generate adaptive dialogue content, while a mixed-methods evaluation framework assesses usability and learning impact. Our key contribution is the first systematic integration of three agent design dimensions: character expressivity, vocal prosody, and cross-cultural adaptability. Experimental results demonstrate significant improvements in user engagement, perceived usability, affective resonance, and self-initiated learning behaviors—particularly among beginner/intermediate learners and those from non-East Asian cultural backgrounds. The findings reveal underlying mechanisms through which agent personification influences learning motivation and strategic language use.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This study investigates how stylized, voiced agents shape user interaction in a multimodal language learning environment. We conducted a mixed-methods evaluation of 54 participants interacting with anime-inspired characters powered by large language models and expressive text-to-speech synthesis. These agents responded in Japanese character language, offering users asynchronous, semi-structured conversation in varying speech styles and emotional tones. We analyzed user engagement patterns, perceived usability, emotional responses, and learning behaviors, with particular attention to how agent stylization influenced interaction across language proficiency levels and cultural backgrounds. Our findings reveal that agent design, especially voice, persona, and linguistic style, substantially affected user experience, motivation, and strategy. This work contributes to the understanding of affective, culturally stylized agents in human-agent interaction and offers guidance for designing more engaging, socially responsive systems.

Problem

• How stylized agents affect user interaction in language learning
• Impact of agent design on user experience and motivation
• Role of cultural stylization in human-agent interaction outcomes

Innovation

• Anime-inspired characters with large language models
• Expressive text-to-speech synthesis in Japanese
• Stylized dialogue for varied speech styles