TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes

2026-06-24

AI Summary

This work addresses the challenge of efficiently teaching dynamic skills—such as a tennis backhand—to humanoid robots without relying on large datasets or intricate reward engineering. The authors propose TaskNPoint, a training protocol that explicitly models the coach–learner paradigm, requiring only a single short video demonstration, discrete skill commands, an interaction window, and target positions from a human instructor. By integrating trajectory optimization in physics-based simulation with stochastic target sampling, the method enables rapid skill acquisition. Trained in under one hour on a single GPU, the system achieves zero-shot generalization to novel targets on the Unitree G1 humanoid robot, successfully executing diverse tasks including forehand and backhand tennis swings, ball kicking, and pick-and-place operations—all without task-specific reward tuning.

Abstract

How do we learn to hit a tennis backhand? Not from a thousand hours of tennis tournaments on TV - we work with a coach and practice. We argue this is also the right recipe for teaching dynamic skills to humanoid robots. This follows from a structural property of dynamic skills: the outcome is decided by a short, crucial portion of the trajectory - for a backhand, the ~20cm of racket travel around ball contact. Getting this interaction window right requires coordinating the whole motion, so that control, physics, and morphology act in concert. Learning thus reduces to mastering a handful of distinct actions and, for each, practicing until the window comes out right. To this end, we introduce TaskNPoint, a training protocol which makes the coach-learner division of labor explicit. The human coach contributes four inputs: a discrete set of skills (e.g. different shots), one demonstration per skill, identification of the interaction window, and the goal. Learning in a physically realistic simulation environment fills in each action trajectory and provides robustness to unmodeled events. Crucially, randomized target sampling during training lets a single demonstration generalize zero-shot to unseen goal locations. We test this approach on a Unitree G1 humanoid that hits forehands and backhands against balls thrown by a human, kicks incoming soccer balls, and picks and places boxes from novel locations. We find that learning is successful from short human video demonstrations and under an hour of training on a single GPU, with no per-task reward tuning.

Problem

• humanoid robots
• dynamic skills
• interaction window
• zero-shot generalization
• skill learning

Innovation

• TaskNPoint
• dynamic skills
• interaction window
• zero-shot generalization
• human-in-the-loop learning