AffAdapt: AFFect-driven ADAPTive AI Personas for Seamless Conversations

2026-08-23

AI Summary

研究提出AffAdapt框架，通过协调语音识别、对话轮次管理、基于角色的响应生成等技术，解决AI角色在多模态环境下的自然流畅对话问题。

Abstract

AI-generated personas are being increasingly used for support, training and simulations. While generative AI models possess abilities to generate affect-aware responses, their embodiment into visual personas is an active area of investigation. Naturalistic exchanges require understanding of the conversational partners' turn completions, whether the agent should respond or keep listening and rely on non-verbal cues aligned with one's emotional states. Seamless human-AI conversation in a multimodal setting requires all modalities being generated to act in coordination. We present AffAdapt, a seamless interaction design framework for AI-personas, which coordinates streaming speech recognition, proactive turn-management, persona-grounded response generation, a persistent emotional state, and synchronized embodied output into a single interaction loop. We demonstrate the architecture in the context of practicing sensitive, high-stakes conversations, and report an initial case study showing fluid turn management and adaptive, persona-consistent behavior, alongside open challenges in interruption handling, open-ended dialogue, and multimodal affective alignment. AffAdapt's interaction loop is a generalizable pattern for coordinating timing, identity, and affect in real-time AI personas - applicable to training, coaching, education, and simulation contexts wherever believable, responsive interaction matters.

Problem

• AI-personas
• seamless conversation
• multimodal interaction
• affective alignment

Innovation

• affect-driven
• adaptive AI personas
• seamless conversations
• multimodal setting
• proactive turn-management