MedVA: An End-to-End Neuro-Symbolic Agentic System for Medical Volume Visualization

2026-09-13

AI Summary

本文提出MedVA系统，通过三个互补代理解决医学体积可视化中ROI选择和视觉强调控制问题，改进了现有基于MLLM方法的局限。

Abstract

Medical volume visualization requires selecting regions of interest (ROIs) and carefully controlling their relative visual emphasis according to a given clinical intent. Implementing these decisions in conventional workflows demands substantial clinical and visualization expertise and often involves trial-and-error optimization. Recent agentic systems have introduced natural-language interaction and autonomous visualization operations but largely rely on MLLM-based inference throughout the workflow. Although MLLMs encode broad medical knowledge and provide strong reasoning capabilities, such inference may be suboptimal for medical volume visualization, potentially leading to clinically incomplete interpretations of user requests and unreliable ROI identification and visualization optimization. In this work, we present MedVA, an end-to-end neuro-symbolic agentic system for medical volume visualization that addresses these limitations through three complementary agents. The neuro-symbolic intent formulation agent refines MLLM-based interpretations of natural-language requests through symbolic reasoning over established clinical knowledge, which provides more complete, clinically grounded ROI specifications than MLLM-only reasoning. The multi-model ROI identification agent directly identifies semantically specified ROIs in the original volume by leveraging complementary large-scale pretrained medical segmentation models. The objective-driven visualization optimization agent explicitly evaluates ROI visibility and occlusion in the original volume using a volume-based visibility objective. Extensive agent-level and system-level evaluations across diverse medical datasets and interaction scenarios support the effectiveness of the individual agents. A formative user study further indicates high usability and practical value among users with different levels of expertise.

Problem

• Medical Volume Visualization
• Regions of Interest (ROIs)
• Clinical Intent
• Visualization Expertise
• MLLM-based Inference

Innovation

• neuro-symbolic intent formulation
• multi-model ROI identification
• objective-driven visualization optimization