Gaussian Random Fields as an Abstract Representation of Patient Metadata for Multimodal Medical Image Segmentation

2025-03-07

AI Summary

To address the underutilization of patient metadata (e.g., age, disease duration) in chronic wound segmentation—particularly diabetic foot ulcers (DFUs)—this work introduces the first framework modeling structured metadata as Gaussian random fields (GRFs) and embedding them into a multimodal image segmentation network. We propose a novel paradigm: “metadata-grouped training + distance-transform-weighted fusion”, wherein dedicated models are trained on subsets of metadata, and their predictions are dynamically ensemble-weighted based on patient feature similarity. Evaluated on the DFU2022 dataset, our method achieves an IoU of 0.4890 (+0.0220) and a Dice score of 0.6137 (+0.0229), demonstrating significant improvement in personalized lesion localization accuracy. All code is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The growing rate of chronic wound occurrence, especially in patients with diabetes, has become a concerning trend in recent years. Chronic wounds are difficult and costly to treat, and have become a serious burden on health care systems worldwide. Chronic wounds can have devastating consequences for the patient, with infection often leading to reduced quality of life and increased mortality risk. Innovative deep learning methods for the detection and monitoring of such wounds have the potential to reduce the impact to both patient and clinician. We present a novel multimodal segmentation method which allows for the introduction of patient metadata into the training workflow whereby the patient data are expressed as Gaussian random fields. Our results indicate that the proposed method improved performance when utilising multiple models, each trained on different metadata categories. Using the Diabetic Foot Ulcer Challenge 2022 test set, when compared to the baseline results (intersection over union = 0.4670, Dice similarity coefficient = 0.5908) we demonstrate improvements of +0.0220 and +0.0229 for intersection over union and Dice similarity coefficient respectively. This paper presents the first study to focus on integrating patient data into a chronic wound segmentation workflow. Our results show significant performance gains when training individual models using specific metadata categories, followed by average merging of prediction masks using distance transforms. All source code for this study is available at: https://github.com/mmu-dermatology-research/multimodal-grf

Problem

• Integrates patient metadata into chronic wound segmentation.
• Improves segmentation accuracy using Gaussian random fields.
• Demonstrates enhanced performance in diabetic foot ulcer analysis.

Innovation

• Gaussian random fields represent patient metadata
• Multimodal segmentation integrates patient data
• Individual models trained on specific metadata categories