WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots

2026-06-15

AI Summary

Generating co-speech gestures on humanoid robots that are synchronized with vocal emphasis while satisfying stringent kinematic and dynamic constraints is highly challenging, particularly due to the difficulty of executing rapid or overlapping motions. This work proposes WaveSync, a hybrid framework that first leverages a large language model to parse spoken dialogue and generate semantic importance waves, then synthesizes feasible gesture trajectories using dynamic movement primitives (DMPs). Synchronization at the word level between gesture peaks and speech emphasis, along with resolution of motion conflicts, is achieved through wavefront optimization. WaveSync is the first approach to integrate semantic importance waves with wavefront optimization, balancing expressiveness, semantic relevance, and motion feasibility while ensuring hardware safety. Evaluated across five conversational scenarios, WaveSync significantly outperforms three baseline methods in both objective synchronization accuracy and subjective naturalness.

Abstract

Expressive co-speech gestures are crucial for natural human-robot interaction, but generating them on physical humanoid robots is difficult because gesture strokes must align with speech emphasis while satisfying strict kinematic and dynamic constraints. Unlike virtual avatars, humanoid robots cannot freely execute rapid or overlapping motions, making word-level synchronization and hardware-safe motion planning a coupled problem. We present \textbf{WaveSync}, a hybrid framework in which a Large Language Model decomposes dialogue responses into structured semantic schemas and assigns per-word importance weights, constructing a continuous Semantic Importance Wave. Gesture trajectories are shaped through Dynamic Movement Primitives, enforcing kinematic feasibility while enhancing expressiveness. A Wavefront Optimization stage aligns peak-to-peak gesture-speech synchronization and resolves residual kinematic violations through gesture-duration compression and forward propagation. Experimental evaluation based on five dialogue scenarios shows that our method achieves high synchronization accuracy and outperforms three baselines in both objective and subjective evaluations. Each component in WaveSync plays a necessary role in producing gestures that are expressive, semantically grounded, and kinematically compliant. The code, resources, and videos are available at \href{https://github.com/pairs-lab/WaveSync}{WaveSync}

Problem

• co-speech gestures
• humanoid robots
• synchronization
• kinematic constraints
• expressive motion

Innovation

• WaveSync
• co-speech gestures
• Semantic Importance Wave
• Dynamic Movement Primitives
• Wavefront Optimization