EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure

2026-05-01

AI Summary

This work addresses the challenge of effectively unlearning knowledge in federated multimodal learning, where cross-modal coupling and entangled client gradient subspaces hinder complete removal of targeted information. Existing methods fail to block cross-modal reconstruction or isolate modality-specific unlearning directions. To overcome this, we propose the first entanglement-aware federated multimodal unlearning framework, which severs cross-modal reconstruction pathways via dual-modal displacement, isolates unlearning-specific update subspaces through cosine-sine decomposition, and suppresses residual drift using a direction-selective unlearning lock. We further uncover three types of residual anchor mechanisms in contrastive unlearning and unify the elimination of anchoring across cross-modal, subspace, and federated update channels. On benchmarks such as Flickr30K, our method achieves unlearning and retention performance (R@1) within only 0.2 and 4.2 points, respectively, of full retraining—significantly outperforming current baselines.

Abstract

Federated Multimodal Learning (FML) trains multimodal models across decentralized clients while keeping their image-text pairs private. However, joint embedding training entangles forgotten knowledge across both modalities and client gradient subspaces, hindering federated unlearning. Previous federated unlearning approaches neither sever the cross-modal reconstruction channel mediated by bilinear coupling nor separate forget-exclusive update directions from those shared with retained clients. We identify an Anchor Principle for federated multimodal contrastive unlearning: forgotten alignments persist through three residual anchors arising from bilinear cross-modal coupling, principal-angle subspace entanglement, and continued federated updates. At the modality level, we show that bilateral displacement of both visual and language branches closes the cross-modal reconstruction channel. Correspondingly, our method addresses subspace entanglement through Cosine--Sine decomposition of client-update subspaces, isolating forget-exclusive directions from retain support. Moreover, we propose a direction-selective Forget Lock that bounds residual drift across rounds. Combining these strategies, we present EASE, an Entanglement-Aware Subspace Excision framework that closes all three anchor channels under a unified design. EASE demonstrates consistent superiority across multiple datasets and unlearning scenarios, for instance, matching the retrain reference to within 0.2 and 4.2 R@1 points on the forget and retain sides under client unlearning on Flickr30K with CLIP-B/32.

Problem

• Federated Unlearning
• Multimodal Learning
• Cross-modal Entanglement
• Subspace Coupling
• Data Privacy

Innovation

• Federated Unlearning
• Multimodal Learning
• Subspace Decomposition
• Cross-modal Entanglement
• Anchor Closure