Privacy Leakage in Federated Learning in Radiology Reports: A Comparative Evaluation of Tokenizer-Driven Privacy Risks

2026-07-15

AI Summary

This study addresses the privacy risks in federated learning for radiology report generation, where gradient updates may inadvertently leak sensitive clinical information. While the impact of tokenizers on such leakage remains unclear, this work systematically evaluates the privacy-preserving capabilities of GPT-2, RadBERT, and LLaMA-2 tokenizers across multiple radiology text datasets under a fixed model architecture. Employing analytical gradient inversion attacks to simulate a malicious server manipulating the architecture, the authors assess reconstruction fidelity using S-BLEU scores and clinical term coverage. Results reveal that none of the tokenizers effectively prevent information leakage, achieving exact sentence reconstruction rates of 31%–44%, with RadBERT yielding the highest fidelity by recovering 18.1% of clinical terms. Although leakage diminishes with larger batch sizes, it remains substantial. This work establishes tokenizer choice as a critical factor influencing textual privacy in federated learning, beyond its conventional role in model performance.

Abstract

Federated learning (FL) enables multi-institutional training on clinical text without sharing raw data, but gradient inversion can reconstruct sensitive information from shared model updates. The extent of this leakage for radiology reports, and the role of tokenizer design, remains unclear. We quantify gradient-based text reconstruction in FL and compare privacy risk across three tokenizers with the model architecture held fixed. Six FL clients trained a GPT-2-style transformer (sequence length 32) on public radiology corpora (368,751 diagnostic reports, 98,206 discharge summaries, 1,500 MIMIC-CXR free-text reports) using the GPT-2, RadBERT, and LLaMA-2 tokenizers at batch sizes of 64, 128, and 256. Assuming an active malicious server that modifies the shared architecture before distribution, we applied analytic gradient inversion and measured reconstruction fidelity over five runs. Exact sentence reconstruction ranged from 31% to 44% across tokenizers (30.6-43.5% across the 27 tokenizer x dataset x batch-size cells). At batch size 64 on the Discharge dataset, accuracy was 42.1% (GPT-2), 42.3% (RadBERT), and 39.4% (LLaMA-2), decreasing to 37.3%, 37.2%, and 34.3% at batch size 256. S-BLEU declined as batch size grew (GPT-2: 0.44 to 0.33; RadBERT: 0.48 to 0.35). RadBERT yielded the highest reconstruction fidelity and recovered the most clinical terms (18.1% of a 1,440-term reference vocabulary, vs 12.5% for GPT-2 and 9.4% for LLaMA-2), yet no tokenizer prevented leakage. Substantial portions of report text are therefore recoverable from FL gradients even at larger batch sizes and with domain-specific tokenizers. Tokenizer design influences leakage severity and is a privacy-relevant decision, not only a utility one; safeguards such as secure aggregation and differential privacy are likely necessary to meet HIPAA and GDPR requirements for FL in radiology NLP.

Problem

• Federated Learning
• Privacy Leakage
• Radiology Reports
• Tokenizer
• Gradient Inversion

Innovation

• federated learning
• gradient inversion attack
• tokenizer design
• privacy leakage
• radiology NLP