VisionCAD: An Integration-Free Radiology Copilot Framework

2025-10-31

AI Summary

Clinical deployment of computer-aided diagnosis (CAD) systems is hindered by their reliance on hospital IT infrastructure integration, particularly with PACS and RIS. Method: This paper proposes a “zero-integration” radiology assistance framework that employs an external camera to capture medical images displayed on clinical monitors. The captured video frames undergo image restoration, object detection, and deep learning–based analysis to enable AI-assisted diagnosis and natural language report generation—bypassing PACS/RIS interfaces entirely and supporting plug-and-play deployment. Contribution/Results: Evaluated on multimodal medical imaging data, the framework achieves classification F1-scores within <2% and report-generation key metric gaps ≤1% relative to conventional CAD systems operating on native digital images. To our knowledge, this is the first work to systematically adopt visual-input paradigms for clinical imaging analysis, offering a lightweight, broadly deployable pathway for AI in healthcare.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Widespread clinical deployment of computer-aided diagnosis (CAD) systems is hindered by the challenge of integrating with existing hospital IT infrastructure. Here, we introduce VisionCAD, a vision-based radiological assistance framework that circumvents this barrier by capturing medical images directly from displays using a camera system. The framework operates through an automated pipeline that detects, restores, and analyzes on-screen medical images, transforming camera-captured visual data into diagnostic-quality images suitable for automated analysis and report generation. We validated VisionCAD across diverse medical imaging datasets, demonstrating that our modular architecture can flexibly utilize state-of-the-art diagnostic models for specific tasks. The system achieves diagnostic performance comparable to conventional CAD systems operating on original digital images, with an F1-score degradation typically less than 2% across classification tasks, while natural language generation metrics for automated reports remain within 1% of those derived from original images. By requiring only a camera device and standard computing resources, VisionCAD offers an accessible approach for AI-assisted diagnosis, enabling the deployment of diagnostic capabilities in diverse clinical settings without modifications to existing infrastructure.

Problem

• Integrating CAD systems with hospital IT is challenging
• VisionCAD captures medical images from displays using cameras
• It transforms camera data into diagnostic-quality images for analysis

Innovation

• Camera captures medical images from displays
• Automated pipeline restores and analyzes images
• Modular architecture uses state-of-the-art diagnostic models