Contribution-Aware Bandwidth Allocation for Multimodal Split Learning

2026-09-01

AI Summary

研究解决了多模态分割学习中带宽分配问题，通过基于Shapley贡献值动态调整每种模态的数据保留率，提高了模型准确性和压缩效率。

Abstract

Multimodal models are increasingly the default option for perception at the network edge, yet they are trained almost entirely in the datacenter, because a client holding several sensor streams cannot host an encoder per modality. Split Learning makes such training feasible by keeping only the first layers on the device, at the cost of an uplink that must carry smashed activations for every modality at every step. Existing compression schemes give each modality the same keep-ratio, so the shared budget is divided in proportion to smashed-activation dimension, a quantity unrelated to how much each modality contributes to the fused prediction. We make that division an explicit decision and call it inter-modality allocation: under a fixed uplink budget, every policy transmits the same expected payload and differs only in how that payload is split across modalities. Our allocator, ModalShare, sets each modality's keep-ratio from a Shapley contribution score that the server computes over coalitions of activations it has already received. Measuring this score adds no uplink traffic and no client-side computation, and needs no prior knowledge of which stream is which. ModalShare improves accuracy over equal keep-ratios by 15.4 and 12.4 percentage points on CREMA-D and MVSA at matched payload in 5x compression, with strong performance across three compressors, three datasets, and four budgets. We show that existing compressors underperform in multimodal settings, with ModalShare recovering what gains are left behind.

Problem

• Multimodal Split Learning
• Bandwidth Allocation
• Contribution-Aware
• Shapley Value
• Uplink Budget

Innovation

• Multimodal Split Learning
• Contribution-Aware Bandwidth Allocation
• Shapley Contribution Score