AdaFuse: Adaptive Multimodal Fusion for Lung Cancer Risk Prediction via Reinforcement Learning

2026-01-30

AI Summary

Existing multimodal fusion approaches typically employ a fixed combination of modalities for all patients, lacking individualized adaptability. This work proposes a novel dynamic fusion framework that, for the first time, formulates patient-level multimodal selection as a sequential decision-making problem using reinforcement learning. The framework adaptively determines—based on individual patient characteristics—whether to incorporate additional modalities (e.g., imaging, clinical notes, radiology reports) or to terminate prediction early. By enabling on-demand fusion, the method achieves an AUC of 0.762 on the NLST dataset, outperforming single-modality models, conventional fixed-fusion strategies, and existing adaptive approaches. Furthermore, it reduces computational overhead while significantly enhancing both the efficiency and accuracy of personalized diagnosis.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multimodal fusion has emerged as a promising paradigm for disease diagnosis and prognosis, integrating complementary information from heterogeneous data sources such as medical images, clinical records, and radiology reports. However, existing fusion methods process all available modalities through the network, either treating them equally or learning to assign different contribution weights, leaving a fundamental question unaddressed: for a given patient, should certain modalities be used at all? We present AdaFuse, an adaptive multimodal fusion framework that leverages reinforcement learning (RL) to learn patient-specific modality selection and fusion strategies for lung cancer risk prediction. AdaFuse formulates multimodal fusion as a sequential decision process, where the policy network iteratively decides whether to incorporate an additional modality or proceed to prediction based on the information already acquired. This sequential formulation enables the model to condition each selection on previously observed modalities and terminate early when sufficient information is available, rather than committing to a fixed subset upfront. We evaluate AdaFuse on the National Lung Screening Trial (NLST) dataset. Experimental results demonstrate that AdaFuse achieves the highest AUC (0.762) compared to the best single-modality baseline (0.732), the best fixed fusion strategy (0.759), and adaptive baselines including DynMM (0.754) and MoE (0.742), while using fewer FLOPs than all triple-modality methods. Our work demonstrates the potential of reinforcement learning for personalized multimodal fusion in medical imaging, representing a shift from uniform fusion strategies toward adaptive diagnostic pipelines that learn when to consult additional modalities and when existing information suffices for accurate prediction.

Problem

• multimodal fusion
• lung cancer risk prediction
• modality selection
• adaptive decision-making
• personalized diagnosis

Innovation

• adaptive multimodal fusion
• reinforcement learning
• lung cancer risk prediction
• sequential decision making
• modality selection