Generating Natural and Expressive Robot Gestures through Iterative Reinforcement Learning with Human Feedback using LLMs

2026-06-17

AI Summary

This work addresses the challenge of generating natural, fluent, and contextually appropriate co-speech gestures for social robots, a task at which traditional animation methods often fall short due to limited flexibility. The authors propose a novel framework that integrates a large language model (LLM) with the humanoid robot Pepper, enabling real-time translation of natural language into executable gesture code. For the first time in this domain, reinforcement learning from human feedback (RLHF) is incorporated to enable closed-loop optimization, allowing the robot to iteratively refine its gestures based on user evaluations. This approach significantly enhances the naturalness, contextual relevance, and fluidity of generated gestures. User studies demonstrate that the RLHF-optimized gestures substantially outperform the initial LLM-generated outputs in terms of expressiveness and overall interaction quality.

Abstract

Expressive gestures are essential for natural and effective communication, complementing speech when verbal cues alone are insufficient (e.g., pointing). For social robots such as the humanoid Pepper, producing natural and expressive movements is critical for improving human-robot interaction (HRI) and long-term acceptance. However, generating gestures remains challenging due to reliance on expert-authored animations, resulting in rigid behaviors that are impractical for dynamic and diverse environments. Alternatively, machine learning approaches often struggle to capture perceived naturalness, becoming increasingly challenging with more degrees of freedom. Consequently, producing expressive robot gestures requires a system that can adapt to the environment while adhering to social norms and physical constraints. Recent advances in large language models (LLMs) enable dynamic code generation, offering new opportunities for runtime gesture synthesis from natural language. In this paper, we integrate ChatGPT into the humanoid robot Pepper to generate co-speech gestures aligned with conversational output. While this baseline enables flexible gesture generation, the resulting motions are often perceived as stiff and unnatural. To address this limitation, we introduce an iterative reinforcement learning with human feedback (RLHF) system that finetunes gesture generation based on user evaluations, leveraging an iterative user study to compare Pepper's generated gestures. Our results show that RLHF improved the LLM's co-speech generative capabilities, producing more expressive, relevant and fluid movements.

Problem

• expressive gestures
• human-robot interaction
• naturalness
• co-speech gestures
• social robots

Innovation

• Iterative Reinforcement Learning with Human Feedback
• Large Language Models
• Co-speech Gestures
• Human-Robot Interaction
• Expressive Robot Motion