Automated Synthesis of Facial Mechanisms for Conversational Animatronic Robots

2026-07-13

AI Summary

Traditional biomimetic robotic faces require manual redesign of mechanical structures for each new facial geometry, hindering personalization and scalability. This work proposes a parametric linkage-driven facial mechanical template coupled with a hierarchical automatic design algorithm that reconstructs a 3D face from a single 2D portrait and automatically generates a collision-free, manufacturable internal actuation mechanism. The approach integrates an anatomy-informed feasible motion space, action-unit-driven expression trajectory optimization, collision-aware outer-loop refinement, and audio-driven real-time facial animation. It achieves, for the first time, fully automatic and scalable mechanical face synthesis adaptable to diverse facial geometries, supporting natural speaking and listening behaviors in bidirectional dialogue. Experiments demonstrate that the generated mechanisms outperform hand-crafted designs, and user studies confirm the naturalness and effectiveness of the resulting animations.

Abstract

Animatronic faces are a central component of socially interactive robots, enabling rich nonverbal communication through facial articulation. However, state-of-the-art animatronic faces are typically tailored systems: each new facial geometry requires extensive manual mechanical redesign, making large-scale personalization prohibitively slow and costly. In this work, we pursue automated and scalable mechanical face synthesis, aiming to rapidly generate a physically realizable facial mechanism for a wide range of facial geometries. We introduce a parametric, linkage-driven mechanical face template whose topology and actuator layout are explicitly parameterized to support systematic scaling and retargeting across diverse facial morphologies. Building on this template, we propose a hierarchical automatic design algorithm that takes a single 2D portrait as input, reconstructs a target 3D face, and synthesizes a collision-free, manufacturable internal mechanism. The algorithm combines anatomy-guided feasible motion volumes, Action Unit (AU)-derived trajectory-based expressiveness objectives, and a collision-driven outer-loop refinement strategy. Beyond hardware synthesis, we argue that future mechanical faces deployed at scale must engage in bidirectional, multi-turn conversation rather than functioning solely as speaking or listening heads. To this end, we develop a dual-identity conversational facial motion synthesis framework that jointly models speaking and listening behaviors from audio, producing temporally coherent 3D facial motion suitable for physical execution. We validate our system through extensive experiments, including (i) quantitative evaluation of automatic mechanism synthesis across diverse facial geometries, (ii) comparisons against manual mechanical design, (iii) benchmarks on conversational facial motion synthesis and real-time deployment, and (iv) perceptual user studies.

Problem

• animatronic faces
• mechanical redesign
• facial geometries
• personalization
• scalable synthesis

Innovation

• automated mechanism synthesis
• parametric facial template
• conversational animatronics
• Action Unit-driven expressiveness
• dual-identity motion synthesis