ViFusion: In-Network Tensor Fusion for Scalable Video Feature Indexing

2025-06-19

AI Summary

To address bandwidth waste and communication overhead bottlenecks caused by high-concurrency small-tensor transfers in large-scale video feature indexing, this paper proposes a communication-aware end-network co-designed tensor fusion paradigm—marking the first effort to offload distributed feature aggregation to the collaborative layer of programmable switches and hosts. Our approach integrates P4 switch programming, RDMA, adaptive multipath routing, and a lightweight tensor serialization-and-fusion algorithm, forming an end-to-end feature compression and transmission stack. Unlike prior host-only optimization schemes, our work overcomes coordination challenges imposed by heterogeneous hardware, multipath network topologies, and diverse application requirements. Evaluated in a real datacenter deployment, it achieves 8–22× higher video retrieval throughput than state-of-the-art systems, while maintaining comparable latency.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Hardware-aware ML
• Computer Vision: Image and Video Retrieval

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Large-scale video feature indexing in datacenters is critically dependent on efficient data transfer. Although in-network computation has emerged as a compelling strategy for accelerating feature extraction and reducing overhead in distributed multimedia systems, harnessing advanced networking resources at both the switch and host levels remains a formidable challenge. These difficulties are compounded by heterogeneous hardware, diverse application requirements, and complex multipath topologies. Existing methods focus primarily on optimizing inference for large neural network models using specialized collective communication libraries, which often face performance degradation in network congestion scenarios. To overcome these limitations, we present ViFusion, a communication aware tensor fusion framework that streamlines distributed video indexing by merging numerous small feature tensors into consolidated and more manageable units. By integrating an in-network computation module and a dedicated tensor fusion mechanism within datacenter environments, ViFusion substantially improves the efficiency of video feature indexing workflows. The deployment results show that ViFusion improves the throughput of the video retrieval system by 8--22 times with the same level of latency as state-of-the-art systems.

Problem

• Efficient data transfer for large-scale video feature indexing
• Challenges in harnessing advanced networking resources effectively
• Performance degradation of existing methods under network congestion

Innovation

• In-network tensor fusion for video indexing
• Communication aware framework consolidates feature tensors
• Integrates in-network computation and tensor fusion