Problem
Research questions and friction points this paper is trying to address.
autism spectrum disorder
behavioral scoring
home videos
early diagnosis
multimodal large language model
Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videos
🤖 AI Summary
This study addresses the critical need for early, accessible autism spectrum disorder (ASD) screening by leveraging home videos to overcome diagnostic delays. It proposes a novel approach that applies Low-Rank Adaptation (LoRA) fine-tuning to the multimodal large language model Gemini 2.5 Pro, using only 30 clinically validated behavioral features extracted from 400 home videos to generate reliable behavioral scores. The method substantially improves agreement with clinician ratings—increasing weighted Cohen’s kappa by 40%—and boosts F1 performance by 53% in zero-shot ASD classification. When integrated into a classifier-assisted pipeline, the system achieves 77% accuracy and an AUC of 86%, demonstrating strong potential for extracting clinical-grade behavioral markers from unstructured home videos to support scalable, early ASD screening.
📝 Abstract
Autism spectrum disorder (ASD) affects 1 in 31 US children, yet median age at diagnosis exceeds four years. Artificial intelligence pipelines that provide quantified diagnosis using easy to access observational data (e.g., home videos) could help with earlier diagnosis, and timely delivery of early treatments. We fine-tuned Gemini 2.5 Pro on 400 clinician-rated home videos with low-rank adaptation, training only on 30 behavioral features previously validated to produce reliable predictions when passed to various ML models. On 99 held-out children (49 ASD, 50 neurotypical), inter-rater reliability with clinicians (per-feature weighted Cohen's kappa) improved by 40% (p<0.001), with 27 of 28 evaluable features improving. As an emergent zero-shot capability, direct ASD diagnosis F1 improved by 53% (p<0.001), matching or exceeding clinician outcomes. Classifier-assisted pipelines using fine-tuned LLM-derived behavioral features matched clinician-scored inputs across all tested pathways and achieved 77% accuracy (95% CI: 68-85%) and an AUC of 86% (95% CI: 78-92%). Fine-tuned multimodal LLMs can serve as scalable behavioral feature extractors for use in autism assessment and diagnosis.
Innovation
Methods, ideas, or system contributions that make the work stand out.
multimodal LLM fine-tuning
low-rank adaptation
autism behavioral scoring
home video analysis
zero-shot diagnosis
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
