MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays

2026-05-14

AI Summary

Existing medical visual question answering benchmarks are largely confined to single or short-term paired chest X-rays, limiting their ability to evaluate models’ capacity for reasoning about longitudinal disease progression. This work proposes MI-CXR—the first multi-visit longitudinal reasoning benchmark that requires neither free-text reports nor multimodal clinical context—constructed from large-scale five-visit chest X-ray sequences and structured as five-choice questions spanning three task types: event localization, interval change reasoning, and global trajectory summarization. Evaluation across 14 state-of-the-art vision-language models yields an average accuracy of only 29.3%, marginally above random chance, systematically revealing critical deficiencies in current models’ temporal consistency and integration of global evidence, thereby underscoring the benchmark’s pivotal role in advancing longitudinal medical visual reasoning research.

Abstract

Longitudinal chest X-ray (CXR) interpretation requires reasoning over disease evolution across multiple patient visits, yet most existing medical VQA benchmarks focus on single images or short-horizon image pairs. We introduce MI-CXR, a benchmark for standardized evaluation of Multi-Interval longitudinal reasoning over multi-visit CXR sequences, without requiring free-form report generation or additional clinical context. MI-CXR comprises five-way multiple-choice questions over five-visit patient timelines and instantiates three complementary task families: Temporal Event Localization, Interval-wise Change Reasoning, and Global Trajectory Summarization, which assess clinically grounded visual reasoning over time. Evaluating 14 state-of-the-art vision-language models (VLMs) shows low overall performance, with an average accuracy of 29.3%, only modestly above random guessing. Using stage-wise diagnostic probing, we find that models often produce locally plausible interval descriptions but fail to enforce temporal constraints or compose evidence into globally consistent decisions over the full timeline. These findings reveal key limitations of current VLMs and establish MI-CXR as a principled benchmark for longitudinal medical reasoning. The benchmark is available at https://github.com/AIDASLab/MI-CXR

Problem

• longitudinal reasoning
• chest X-ray
• medical VQA
• multi-interval
• temporal reasoning

Innovation

• longitudinal reasoning
• multi-interval CXR
• vision-language models
• temporal consistency
• medical VQA