Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence

2026-08-13

AI Summary

This study addresses the challenge that existing vision-language models struggle to effectively integrate visual and textual information in Polish medical visual question answering (VQA), often over-relying on question text while neglecting image evidence. The authors present the first multi-specialty medical VQA benchmark derived from Polish physician and dentist certification exams, comprising both image-based questions and a text-only control set, along with a novel method for classifying image importance. Through systematic ablation studies—removing either images or questions—and answer-option analyses on both open-weight and commercial models, they evaluate visual grounding capabilities and reasoning biases. The best-performing model achieves 79.0% accuracy on the full test set, with only GPT-5.6 surpassing human performance on certain subsets. Models consistently underperform on image-dependent questions and can significantly exceed random guessing using answer options alone.

Abstract

We introduce a Polish-language medical visual question answering (VQA) benchmark, built from Polish Board Certification Examination questions for licensed physicians and dentists pursuing specialist certification. The benchmark comprises image-containing questions spanning diverse medical specialties and visual domains, together with a text-only question answering (QA) control set. We evaluate Polish-oriented, general-purpose open-weight, and commercial vision-language models. The task remains challenging: the best model achieves 79.0\% accuracy on the full VQA set, and only GPT-5.6 surpasses the approximate human reference on the subset with available candidate responses; all other evaluated models perform worse than humans. To assess visual grounding, we compare complete inputs with configurations omitting the image, the question, or both, and categorize questions by image importance. Models derive more useful information from the question text than from the image and perform worse on image-dominant questions. Across both QA and VQA, they nevertheless achieve above-chance accuracy from the answer choices alone, showing that non-trivial performance can persist even when key task components are missing.

Problem

• Medical Visual Question Answering
• Vision-Language Models
• Visual Grounding
• Polish-language VQA
• Image Importance

Innovation

• medical visual question answering
• vision-language models
• Polish-language benchmark
• visual grounding
• image-dominant questions