Digitizing Coaching Intelligence: An Agentic Framework for Holistic Athlete Profiling using VLM and RAG

2026-06-26

AI Summary

This work addresses the limitations of traditional athlete assessment methods, which rely on subjective observation or simplistic visual counting systems that lack scalability and the ability to intelligently evaluate movement quality or physiological indicators such as fatigue. To overcome these challenges, the authors propose a large language model–based hybrid agent framework that integrates computer vision and vision-language models, strictly adhering to the Sports Authority of India’s evaluation protocols to enable automated, multidimensional athlete profiling. Key innovations include a 3×3 “smart grid” temporal chunking strategy that reduces computational overhead by 88%, an “LLM-as-a-Judge” self-correction mechanism, and a dual-persistent RAG architecture supporting natural language queries. Built upon MediaPipe, Llama-4-scout, and LangGraph, the system effectively bridges biomechanical data with actionable coaching insights, offering a scalable and objective intelligent assessment solution for national-level talent identification.

Abstract

Athlete assessment is a critical process for tracking physical progress and identifying elite talent. However, during mass recruitment drives, traditional methods rely on manual observation, which is inherently subjective and unscalable, or basic computer vision (CV) systems limited to quantitative repetition counting. These standard approaches lack the "coaching intelligence" required to evaluate qualitative physiological markers such as form degradation, spinal articulation, and fatigue. This paper presents a novel, LLM-based hybrid agentic framework for automated, holistic athlete profiling that strictly aligns with the Sports Authority of India (SAI) assessment protocols. Orchestrated via LangGraph, our dual-pipeline architecture synthesizes the geometric precision of CV (MediaPipe) for kinematic tracking with the semantic reasoning of Vision-Language Models (Llama-4-scout). To overcome the latency and token constraints associated with multimodal video processing, we introduce a 3 X 3 "Smart Grid" temporal chunking strategy, reducing computational overhead by over 88% while preserving critical temporal continuity. To ensure data integrity and mitigate hallucination, the framework pioneers an autonomous "LLM-as-a-Judge" self-correction loop that cross-references quantitative and qualitative metrics before persistence. Finally, we implement a dual-persistence Retrieval-Augmented Generation (RAG) pipeline utilizing a vector search engine (ChromaDB). This enables coaches to bypass rigid SQL databases and perform complex semantic queries (e.g., "Identify athletes with high endurance but poor core rigidity") using natural language. Experimental results demonstrate that this multi-agent approach significantly bridges the gap between raw biometric tracking and actionable coaching insights, offering a scalable, objective solution for national talent identification.

Problem

• athlete profiling
• coaching intelligence
• qualitative physiological markers
• scalable assessment
• subjective evaluation

Innovation

• Vision-Language Model
• Retrieval-Augmented Generation
• Agentic Framework
• Temporal Chunking
• LLM-as-a-Judge