Understanding Cross Task Generalization in Handwriting-Based Alzheimer's Screening via Vision Language Adaptation

2025-11-08

AI Summary

Early Alzheimer’s disease (AD) screening via handwriting analysis suffers from unclear task-type effects and poor cross-task generalization. Method: This study systematically investigates how distinct handwriting tasks influence AD classification performance and proposes a lightweight cross-layer fusion adapter framework enabling prompt-free, zero-shot inference with CLIP on handwritten medical images. The method integrates the visual encoder with multi-level adapters to bridge handwritten images and semantic language modalities, supporting zero-shot anomaly detection and cross-task transfer analysis. Contribution/Results: Experiments reveal discriminative stroke patterns and task-specific features for early AD detection. We introduce the first handwriting-based cognitive assessment benchmark and achieve significant improvements in diagnostic consistency across tasks—average cross-task accuracy increases by 12.7%. This work establishes a novel, non-invasive paradigm for AD screening grounded in behavioral biomarkers.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Alzheimer's disease is a prevalent neurodegenerative disorder for which early detection is critical. Handwriting-often disrupted in prodromal AD-provides a non-invasive and cost-effective window into subtle motor and cognitive decline. Existing handwriting-based AD studies, mostly relying on online trajectories and hand-crafted features, have not systematically examined how task type influences diagnostic performance and cross-task generalization. Meanwhile, large-scale vision language models have demonstrated remarkable zero or few-shot anomaly detection in natural images and strong adaptability across medical modalities such as chest X-ray and brain MRI. However, handwriting-based disease detection remains largely unexplored within this paradigm. To close this gap, we introduce a lightweight Cross-Layer Fusion Adapter framework that repurposes CLIP for handwriting-based AD screening. CLFA implants multi-level fusion adapters within the visual encoder to progressively align representations toward handwriting-specific medical cues, enabling prompt-free and efficient zero-shot inference. Using this framework, we systematically investigate cross-task generalization-training on a specific handwriting task and evaluating on unseen ones-to reveal which task types and writing patterns most effectively discriminate AD. Extensive analyses further highlight characteristic stroke patterns and task-level factors that contribute to early AD identification, offering both diagnostic insights and a benchmark for handwriting-based cognitive assessment.

Problem

• Investigating how handwriting task types affect Alzheimer's diagnostic performance and generalization
• Adapting vision-language models for zero-shot handwriting-based Alzheimer's screening without retraining
• Identifying characteristic stroke patterns and task factors for early Alzheimer's detection

Innovation

• Lightweight adapter framework repurposes CLIP model
• Multi-level fusion aligns handwriting-specific medical cues
• Enables prompt-free zero-shot inference across tasks