iMedImage Technical Report

2025-03-27

AI Summary

Chromosome karyotyping is critical for genetic disorder diagnosis, yet structural abnormality detection remains hampered by limited accuracy and insufficient automation. To address this, we propose iMedImage—a novel end-to-end multimodal medical foundation model that unifies representation learning across seven imaging modalities: chromosomes, cytology, histopathology, ultrasound, X-ray, CT, and MRI. iMedImage enables fully automated chromosome segmentation, karyotype assembly, and structural abnormality identification. Its architecture features multi-granularity (case-, image-, and patch-level) recognition, integrated chain-of-thought (CoT) embeddings, and a Mixture-of-Experts (MoE) mechanism to achieve cross-modal representation alignment and differentiable pipeline modeling. Evaluated on real-world data from 12 clinical institutions, iMedImage achieves 92.75% sensitivity and 91.5% specificity—marking substantial improvements in screening efficiency and diagnostic consistency for genetic disorders.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Background: Chromosome karyotype analysis is crucial for diagnosing hereditary diseases, yet detecting structural abnormalities remains challenging. While AI has shown promise in medical imaging, its effectiveness varies across modalities. Leveraging advances in Foundation Models that integrate multimodal medical imaging for robust feature extraction and accurate diagnosis, we developed iMedImage, an end-to-end model for general medical image recognition, demonstrating strong performance across multiple imaging tasks, including chromosome abnormality detection. Materials and Methods: We constructed a comprehensive medical image dataset encompassing multiple modalities from common medical domains, including chromosome, cell, pathology, ultrasound, X-ray, CT, and MRI images. Based on this dataset, we developed the iMedImage model, which incorporates the following key features: (1) a unified representation method for diverse modality inputs and medical imaging tasks; (2) multi-level (case-level, image-level, patch-level) image recognition capabilities enhanced by Chain of Thought (CoT) embedding and Mixture of Experts (MoE) strategies. Results: The test set comprised data from 12 institutions across six regions in China, covering three mainstream scanning devices, and included naturally distributed, unscreened abnormal cases. On this diverse dataset, the model achieved a fully automated chromosome analysis workflow, including segmentation, karyotyping, and abnormality detection, reaching a sensitivity of 92.75% and a specificity of 91.5%. Conclusion: We propose iMedImage, an end-to-end foundation model for medical image analysis, demonstrating its superior performance across various medical imaging tasks. iMedImage provides clinicians with a precise imaging analysis tool and contributes to improving diagnostic accuracy and disease screening.

Problem

• Detecting chromosome structural abnormalities for hereditary disease diagnosis
• Developing a unified AI model for diverse medical imaging modalities
• Achieving high accuracy in automated chromosome analysis workflow

Innovation

• Unified representation for diverse medical modalities
• Multi-level recognition with CoT and MoE
• End-to-end automated chromosome analysis workflow