Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

2026-02-24

AI Summary

This work addresses the poor generalization of segmentation models in federated learning when applied to unimodal medical images—such as CT or MRI—due to cross-modal domain shifts. To tackle this challenge without requiring paired multimodal data or complex model architectures, the authors propose a practical augmentation strategy based on Global Intensity Nonlinearity (GIN). This approach combines convolutional spatial transformations, frequency-domain operations, and domain-specific normalization to simulate inter-modal appearance variations while preserving anatomical structures. Extensive experiments under both federated and centralized training paradigms demonstrate that the method significantly improves pancreatic segmentation performance, increasing the Dice score from 0.073 to 0.437—a 498% relative gain. Moreover, the federated implementation achieves 93–98% of the performance of its centralized counterpart, enabling effective cross-modal generalization under strict privacy constraints.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Artificial intelligence has emerged as a transformative tool in medical image analysis, yet developing robust and generalizable segmentation models remains difficult due to fragmented, privacy-constrained imaging data siloed across institutions. While federated learning (FL) enables collaborative model training without centralizing data, cross-modality domain shifts pose a critical challenge, particularly when models trained on one modality fail to generalize to another. Many existing solutions require paired multimodal data per patient or rely on complex architectures, both of which are impractical in real clinical settings. In this work, we consider a realistic FL scenario where each client holds single-modality data (CT or MRI), and systematically investigate augmentation strategies for cross-modality generalization. Using abdominal organ segmentation and whole-heart segmentation as representative multi-class and binary segmentation benchmarks, we evaluate convolution-based spatial augmentation, frequency-domain manipulation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation. Our results show that GIN consistently outperforms alternatives in both centralized and federated settings by simulating cross-modality appearance variations while preserving anatomical structure. For the pancreas, Dice score improved from 0.073 to 0.437, a 498% gain. Our federated approach achieves 93-98% of centralized training accuracy, demonstrating strong cross-modality generalization without compromising data privacy, pointing toward feasible federated AI deployment across diverse healthcare systems.

Problem

• federated learning
• cross-modality
• medical image segmentation
• domain shift
• data privacy

Innovation

• Federated Learning
• Cross-Modality Generalization
• Medical Image Segmentation
• Data Augmentation
• Global Intensity Nonlinear