Clinical-Prior Guided Multi-Modal Learning with Latent Attention Pooling for Gait-Based Scoliosis Screening

2026-02-06

AI Summary

This study addresses the limitations of traditional adolescent idiopathic scoliosis (AIS) screening—namely its subjectivity and poor scalability—as well as the data leakage and limited interpretability of existing gait-video-based approaches. To this end, the authors introduce ScoliGait, a benchmark dataset comprising 1,572 training videos and 300 fully independent test videos, and propose a multimodal learning framework that integrates video, text, and a clinical kinematic prior knowledge graph. By designing a latent attention pooling mechanism to effectively fuse heterogeneous information sources, the method achieves a new state-of-the-art performance under a realistic, non-overlapping subject setting. The approach not only significantly enhances model generalization but also provides clinically interpretable decision rationales, establishing a practical foundation for accurate, non-invasive, and scalable AIS screening.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Adolescent Idiopathic Scoliosis (AIS) is a prevalent spinal deformity whose progression can be mitigated through early detection. Conventional screening methods are often subjective, difficult to scale, and reliant on specialized clinical expertise. Video-based gait analysis offers a promising alternative, but current datasets and methods frequently suffer from data leakage, where performance is inflated by repeated clips from the same individual, or employ oversimplified models that lack clinical interpretability. To address these limitations, we introduce ScoliGait, a new benchmark dataset comprising 1,572 gait video clips for training and 300 fully independent clips for testing. Each clip is annotated with radiographic Cobb angles and descriptive text based on clinical kinematic priors. We propose a multi-modal framework that integrates a clinical-prior-guided kinematic knowledge map for interpretable feature representation, alongside a latent attention pooling mechanism to fuse video, text, and knowledge map modalities. Our method establishes a new state-of-the-art, demonstrating a significant performance gap on a realistic, non-repeating subject benchmark. Our approach establishes a new state of the art, showing a significant performance gain on a realistic, subject-independent benchmark. This work provides a robust, interpretable, and clinically grounded foundation for scalable, non-invasive AIS assessment.

Problem

• Adolescent Idiopathic Scoliosis
• gait analysis
• data leakage
• clinical interpretability
• scoliosis screening

Innovation

• multi-modal learning
• latent attention pooling
• clinical prior
• gait analysis
• scoliosis screening