CaddieSet: A Golf Swing Dataset with Human Joint Features and Ball Information

2025-08-28

AI Summary

Existing research lacks quantitative modeling of the relationship between golf swing kinematics and ball trajectory, hindering personalized technique refinement. To address this, we introduce the first multimodal golf swing dataset integrating human joint motion—extracted from single-view videos and segmented into eight biomechanically meaningful phases—with comprehensive ball flight parameters. We further propose 15 expert-derived, interpretable pose metrics grounded in golf biomechanics to enable principled swing performance analysis. Through multimodal fusion modeling, we empirically validate strong domain consistency between joint-level features and ball trajectory prediction. Experiments demonstrate that our dataset supports diverse baseline models for accurate trajectory forecasting; moreover, model-derived feedback aligns closely with established golf coaching principles. The dataset and methodology exhibit strong potential for clinical swing training applications and deployment in intelligent coaching systems.

Technology Category

• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Recent advances in deep learning have led to more studies to enhance golfers' shot precision. However, these existing studies have not quantitatively established the relationship between swing posture and ball trajectory, limiting their ability to provide golfers with the necessary insights for swing improvement. In this paper, we propose a new dataset called CaddieSet, which includes joint information and various ball information from a single shot. CaddieSet extracts joint information from a single swing video by segmenting it into eight swing phases using a computer vision-based approach. Furthermore, based on expert golf domain knowledge, we define 15 key metrics that influence a golf swing, enabling the interpretation of swing outcomes through swing-related features. Through experiments, we demonstrated the feasibility of CaddieSet for predicting ball trajectories using various benchmarks. In particular, we focus on interpretable models among several benchmarks and verify that swing feedback using our joint features is quantitatively consistent with established domain knowledge. This work is expected to offer new insight into golf swing analysis for both academia and the sports industry.

Problem

• Quantifying relationship between golf swing posture and ball trajectory
• Providing interpretable swing feedback using joint features
• Predicting ball trajectories from swing phase analysis

Innovation

• Dataset with joint and ball information
• Computer vision for swing phase segmentation
• Interpretable models using expert-defined metrics