MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution

2026-04-29

AI Summary

Current medical vision-language models struggle to emulate clinicians’ cognitive processes of invoking implicit diagnostic memory due to quantization loss from discrete tokenization, long-range information decay, and a lack of case-adaptive expert knowledge. To address these limitations, this work proposes a latent-variable diagnostic memory evolution framework that dynamically synthesizes implicit memory within the model’s hidden representations, thereby integrating visual perception with clinical intuition. The approach introduces three key innovations: a novel Meta Query for Prior Memorization mechanism, a dual-branch paradigm combining Causal Counterfactual Refinement (CCR) and Endogenous Memory Transfer (IMT) to internalize external expert knowledge into model parameters, and a synergistic integration of learnable anatomical prior probes, reinforcement learning–driven causal rewards, and full-vocabulary distribution alignment. Experiments demonstrate that the proposed method significantly outperforms state-of-the-art models across multiple medical imaging benchmarks, notably surpassing chain-of-thought paradigms in diagnostic accuracy.

Abstract

High-precision medical diagnosis relies not only on static imaging features but also on the implicit diagnostic memory experts instantly invoke during image interpretation. We pinpoint a fundamental cognitive misalignment in medical VLMs caused by discrete tokenization, leading to quantization loss, long-range information dissipation, and missing case-adaptive expertise. To bridge this gap, we propose ours, a framework for latent diagnostic memory evolution that simulates the experiential invocation of clinicians by dynamically synthesizing implicit diagnostic memories within the model's hidden stream. Specifically, it begins with a Meta Query for Prior Memorization mechanism, where learnable probes retrieve structured priors from an anatomical prior encoder to generate condensed implicit memories. To ensure clinical fidelity, we introduce Causal Counterfactual Refinement (CCR), which leverages reinforcement learning and counterfactual rewards derived from region-level feature masking to quantify the causal contribution of each memory, thereby pruning redundancies and aligning latent representations with diagnostic logic. This evolutionary process culminates in Intrinsic Memory Transition (IMT), a privileged-autonomous dual-branch paradigm that internalizes teacher-branch diagnostic patterns into the student-branch via full-vocabulary divergence alignment. Comprehensive empirical evaluations across multiple datasets demonstrate that ours, by transferring external expertise into endogenous parameters, significantly outperforms existing state-of-the-art methods, particularly chain-of-thought paradigms, in diagnostic accuracy.

Problem

• medical VLMs
• diagnostic memory
• cognitive misalignment
• implicit expertise
• quantization loss

Innovation

• Latent Memory Evolution
• Causal Counterfactual Refinement
• Meta Query for Prior Memorization
• Intrinsic Memory Transition
• Medical Visual Language Models