TMF-RSE: Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty for Lung Severity Scoring

2026-07-07

AI Summary

This study addresses the critical need for precise quantification of pulmonary disease severity in chest imaging to support clinical decision-making and resource allocation. The authors propose a novel trimodal fusion architecture that, for the first time, jointly integrates 2D image appearance, structural priors from lung segmentation masks, and semantic information from a vision-language model. Complementary features across modalities are fused through a hierarchical cross-modal interaction mechanism, while evidential regression is introduced to simultaneously predict severity scores and their associated uncertainty estimates. Evaluated on the Per-COVID-19 CT and RALO datasets, the method achieves mean absolute errors (MAE) of 4.02 (Pearson correlation: 0.9629) and 0.339 (Pearson correlation: 0.973), respectively, significantly outperforming existing Transformer-based baselines and demonstrating high accuracy and reliability.

Abstract

Accurate quantification of lung disease severity from chest imaging is critical for clinical decision-making and resource allocation. We propose a tri-modal deep learning framework, TMF-RSE (Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty), that combines appearance features from two-dimensional chest inputs, structural features from lung segmentation masks, and semantic features from vision-language models (VLMs) for severity quantification. Our approach employs complementary fusion mechanisms that integrate semantic guidance, structural priors, and hierarchical interactions across modalities. The model employs evidential regression to provide both severity predictions and uncertainty estimates. Experiments on the Per-COVID-19 CT and RALO datasets show that TMF-RSE outperforms recent transformer-based baselines, achieving MAE of 4.02 and Pearson correlation of 0.9629 on Per-COVID-19 validation, and 0.339 MAE / 0.973 PC on RALO geographic extent.

Problem

• lung severity scoring
• chest imaging
• disease quantification
• clinical decision-making
• resource allocation

Innovation

• Tri-Modal Fusion
• Regional Semantics
• Evidential Uncertainty
• Vision-Language Models
• Lung Severity Scoring