LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering

2026-05-10

AI Summary

This work addresses the limited multi-step reasoning capability of existing compact vision-language models in medical visual question answering, which hinders interpretable clinical decision support. For the first time, the authors distill the full chain-of-thought (CoT) reasoning ability of a 235B teacher model into a 2B student model under a no-image-caption setting, leveraging parameter-efficient LoRA fine-tuning while preserving strong visual grounding. By integrating CoT knowledge distillation with explanation-augmented training data, the proposed method achieves 64.9% accuracy on the PMC-VQA benchmark—outperforming the zero-shot Qwen3-VL-4B baseline by 11 percentage points and surpassing all previously published approaches.

Abstract

The reasoning gap between large and compact vision-language models (VLMs) limits the deployment of medical AI on portable clinical devices. Compact VLMs of 2--4B parameters can run on resource-constrained hardware but lack the multi-step reasoning capacity needed for interpretable clinical decision support. Existing knowledge distillation methods transfer answers without the reasoning process behind them. Medical visual question answering (VQA) serves as a testbed for this problem, as it requires models to integrate visual evidence with clinical knowledge through structured reasoning chains. We introduce LiteMedCoT-VL, a pipeline that transfers chain-of-thought reasoning from a 235B teacher model to 2B student models through LoRA-based fine-tuning on explanation-enriched training data. All inference is conducted without image captions by default, simulating the clinical scenario in which a physician interprets a medical image directly without an accompanying radiology report. On the PMC-VQA benchmark, LiteMedCoT-VL achieves 64.9% accuracy, exceeding the zero-shot Qwen3-VL-4B baseline of 53.9% by 11.0 percentage points and outperforming all published baselines. This result indicates that a 2B model with reasoning distillation can match or exceed models with twice the parameters. Visual grounding analysis shows that the model relies on image content rather than exploiting textual priors. Our code is publicly available at https://anonymous.4open.science/r/LiteMedCoT-VL.

Problem

• medical visual question answering
• compact vision-language models
• multi-step reasoning
• parameter-efficient adaptation
• clinical decision support

Innovation

• chain-of-thought distillation
• parameter-efficient adaptation
• medical visual question answering
• LoRA fine-tuning
• reasoning transfer