Rethinking Diffusion-Based Image Generators for Fundus Fluorescein Angiography Synthesis on Limited Data

2024-12-17

arXiv.org

AI Summary

Addressing the challenges of generating high-quality fundus fluorescein angiography (FFA) images from limited samples, poor cross-disease and cross-modal generalization of existing methods, and the invasive nature of conventional FFA, this paper introduces the first latent diffusion model framework tailored for few-shot medical image synthesis. We propose a novel conditional fine-tuning protocol, a cross-modal feature alignment mechanism, and disease-aware conditional encoding—enabling a single unified model to synthesize anatomically accurate FFA images across multiple pathologies (e.g., diabetic retinopathy, glaucoma) and diverse input modalities (e.g., OCT, color fundus photography). Evaluated on scarce clinical data, our method achieves state-of-the-art performance, significantly improving anatomical fidelity and clinical utility of generated FFA images. The source code will be made publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Fundus imaging is a critical tool in ophthalmology, with different imaging modalities offering unique advantages. For instance, fundus fluorescein angiography (FFA) can accurately identify eye diseases. However, traditional invasive FFA involves the injection of sodium fluorescein, which can cause discomfort and risks. Generating corresponding FFA images from non-invasive fundus images holds significant practical value but also presents challenges. First, limited datasets constrain the performance and effectiveness of models. Second, previous studies have primarily focused on generating FFA for single diseases or single modalities, often resulting in poor performance for patients with various ophthalmic conditions. To address these issues, we propose a novel latent diffusion model-based framework, Diffusion, which introduces a fine-tuning protocol to overcome the challenge of limited medical data and unleash the generative capabilities of diffusion models. Furthermore, we designed a new approach to tackle the challenges of generating across different modalities and disease types. On limited datasets, our framework achieves state-of-the-art results compared to existing methods, offering significant potential to enhance ophthalmic diagnostics and patient care. Our code will be released soon to support further research in this field.

Problem

• Generate FFA images from non-invasive fundus imaging.
• Overcome limited datasets for medical image synthesis.
• Improve FFA generation across multiple diseases and modalities.

Innovation

• Latent diffusion model for FFA synthesis
• Fine-tuning protocol for limited medical data
• Cross-modality and multi-disease generation approach