UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignment

2026-05-08

AI Summary

This work addresses the challenges of modality heterogeneity, distribution shift, and structural signal degradation caused by privacy-preserving noise in device-agnostic localization under federated learning with multimodal heterogeneous sensors. To this end, we propose a graph-based federated learning framework that models clients as nodes in a global graph and aggregates signals in the spectral domain through shared continuous integral operators. Key innovations include a spectral-gated attention mechanism coupled with a diffusion-driven operator alignment strategy to achieve cross-modal low-rank subspace alignment, an anisotropic differential privacy mechanism that safeguards dominant feature directions, and an integrated architecture combining linear attention, low-rank spectral filtering, and a spectral coefficient diffusion model. Experiments on the MM-Fi and RELI11D benchmarks demonstrate significant improvements in localization accuracy, convergence speed, and communication efficiency under high heterogeneity and stringent privacy constraints.

Abstract

Device-free localization trains models from heterogeneous wireless and visual sensors (e.g., Wi-Fi, LiDAR) distributed across edge devices. Federated learning offers a privacy-respecting framework, but is brittle when clients differ in sensor modality and resolution, when their data distributions drift, and when privacy noise destroys the structural signal needed for localization. We propose UMEDA, a graph federated learning framework in which clients form nodes of a global graph that share a continuous integral operator, and aggregation is reformulated as spectral signal processing on this operator. Each client encodes its local sensors with a linear-attention layer whose kernel spectrum is low-rank filtered, suppressing modality-specific residuals so clients with different sensors align in a common low-rank subspace. The server then aggregates client updates via a diffusion model over the kernel's spectral coefficients, treating updates as discretizations of a shared operator rather than topology-bound weights -- this absorbs varying graph sizes and missing modalities without node-wise correspondence. To balance privacy and utility, we add an anisotropic differential-privacy mechanism that projects noise preferentially into the null space of the signal subspace, preserving dominant eigendirections while ensuring formal $(ε, δ)$-DP under gradient clipping. On MM-Fi and the RELI11D out-of-distribution benchmark, UMEDA outperforms state-of-the-art federated baselines in accuracy, convergence, and communication efficiency, particularly under high modality heterogeneity and tight privacy budgets.

Problem

• federated learning
• modality heterogeneity
• privacy-preserving
• graph neural networks
• device-free localization

Innovation

• Spectral-Gated Attention
• Diffusion-Based Operator Alignment
• Graph Federated Learning
• Anisotropic Differential Privacy
• Multi-modal Data Fusion