BioHuman: Learning Biomechanical Human Representations from Video

2026-05-14

AI Summary

Existing approaches struggle to directly infer internal human biomechanical states from monocular video and are hindered by the absence of large-scale datasets annotated with muscle activations. This work proposes the first end-to-end framework that leverages physics-based simulation to generate muscle activation labels, enabling the creation of BioHuman10M—a large-scale dataset comprising synchronized video, kinematics, and muscle activations. Using this dataset, the model is trained to jointly predict 3D human pose and muscle activity directly from in-the-wild video. To our knowledge, this is the first method capable of reconstructing muscle activations solely from visual input, thereby establishing a direct link between external visual observations and internal biomechanics. The approach demonstrates strong generalization across diverse subjects and motions, setting a new benchmark for video-driven biomechanical analysis.

Abstract

Understanding human motion beyond surface kinematics is crucial for motion analysis, rehabilitation, and injury risk assessment. However, progress in this domain is limited by the lack of large-scale datasets with biomechanical annotations, and by existing approaches that cannot directly infer internal biomechanical states from visual observations. In this paper, we introduce a simulation-based framework for estimating muscle activations from existing motion capture datasets, resulting in BioHuman10M, a large-scale dataset with synchronized video, motion, and activations. Building on BioHuman10M, we propose BioHuman, an end-to-end model that takes monocular video as input and jointly predicts human motion and muscle activations, effectively bridging visual observations and internal biomechanical states. Extensive experiments demonstrate that BioHuman enables accurate reconstruction of both kinematic motion and muscle activity, and generalizes across diverse subjects and motions. We believe our approach establishes a new benchmark for video-based biomechanical understanding and opens up new possibilities for physically grounded human modeling.

Problem

• biomechanics
• muscle activation
• human motion
• video-based analysis
• biomechanical annotation

Innovation

• biomechanical representation
• muscle activation estimation
• video-based motion analysis
• simulation-based dataset
• end-to-end human modeling