ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images

2024-06-20

AI Summary

Longitudinal medical imaging data are often irregularly sampled and high-dimensional yet sparse, posing significant challenges for patient-level, image-scale disease progression modeling. Method: We propose the first position-parameterized neural ODE/SDE framework for flow-field modeling, integrating a U-Net–based multi-scale encoder with latent-space organization to jointly represent deterministic dynamics and stochastic variability in a unified latent space. Manifold constraints and high-level visual feature regularization are incorporated to enhance generalizability and interpretability. Results: Evaluated on three real-world datasets—geographic atrophy, multiple sclerosis, and glioblastoma—the method significantly outperforms state-of-the-art approaches, yielding high-fidelity, spatiotemporally consistent, and clinically interpretable image-level predictions of disease progression trajectories at the individual-patient level.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Advances in medical imaging technologies have enabled the collection of longitudinal images, which involve repeated scanning of the same patients over time, to monitor disease progression. However, predictive modeling of such data remains challenging due to high dimensionality, irregular sampling, and data sparsity. To address these issues, we propose ImageFlowNet, a novel model designed to forecast disease trajectories from initial images while preserving spatial details. ImageFlowNet first learns multiscale joint representation spaces across patients and time points, then optimizes deterministic or stochastic flow fields within these spaces using a position-parameterized neural ODE/SDE framework. The model leverages a UNet architecture to create robust multiscale representations and mitigates data scarcity by combining knowledge from all patients. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We validate ImageFlowNet on three longitudinal medical image datasets depicting progression in geographic atrophy, multiple sclerosis, and glioblastoma, demonstrating its ability to effectively forecast disease progression and outperform existing methods. Our contributions include the development of ImageFlowNet, its theoretical underpinnings, and empirical validation on real-world datasets. The official implementation is available at https://github.com/KrishnaswamyLab/ImageFlowNet.

Problem

• Medical Image Analysis
• Irregular Time-series
• Disease Progression Prediction

Innovation

• ImageFlowNet
• multi-scale image pathway prediction
• small data optimization