Task-Oriented Communication for Human Action Understanding via Edge-Cloud Co-Inference

2026-05-08

AI Summary

This work addresses the challenges of high bandwidth consumption, latency, and privacy risks in human action understanding on edge devices by proposing an edge-cloud collaborative inference framework. The approach leverages monocular pose estimation to extract skeletal keypoints and employs a Vector-Quantized Variational Autoencoder (VQ-VAE) to compress actions into discrete motion tokens at an extremely low bitrate. These tokens are then aligned with the embedding space of a large vision-language model via a lightweight projector, enabling task-oriented, efficient communication. Through instruction tuning, the system achieves comparable action recognition accuracy to full video-based methods while transmitting only approximately 1% of the original data volume and reducing end-to-end latency to 20% of baseline levels, effectively balancing efficiency, privacy, and performance.

Abstract

The expanding application of smart sensing has created a growing demand for the accurate understanding of human action at the network edge. Traditional approaches require massive video data to be transmitted from resource-constrained edge devices to powerful cloud servers, incurring prohibitive uplink bandwidth consumption and unacceptable latency while raising privacy concerns. To overcome these bottlenecks, we propose a task-oriented communication framework for human action understanding (TOAU) through edge-cloud collaboration. Our framework utilizes a monocular pose estimator to extract continuous joint coordinates from raw videos, followed by a vector quantized variational autoencoder (VQ-VAE) to convert these coordinates into discrete motion tokens. Consequently, only a compact sequence of codebook indices is transmitted over the network, consuming as few as 9 bits per frame and avoiding privacy leakages. At the cloud server, a lightweight projector aligns these motion tokens with the embedding space of a large vision-language model (VLM) to facilitate complex action understanding, which is trained with an efficient instruction tuning paradigm. Comprehensive evaluations on three benchmarks demonstrate that our TOAU system reduces the transmission payload to approximately 1\% and the system latency to around 20\% compared to video codec-based solutions, while delivering comparable action understanding accuracy.

Problem

• task-oriented communication
• human action understanding
• edge-cloud collaboration
• bandwidth consumption
• privacy concerns

Innovation

• task-oriented communication
• edge-cloud co-inference
• VQ-VAE
• motion tokens
• vision-language model