🤖 AI Summary
To address remote control challenges under uplink-constrained scenarios—such as those involving Reduced-Capability (RedCap) devices and large-scale sensor networks—this paper proposes the Temporal Joint Embedding Predictive Architecture (TS-JEPA) and a lightweight semantic actuator, marking the first application of the JEPA paradigm to temporal teleoperation control. Instead of transmitting raw sensor data, TS-JEPA employs self-supervised learning to model spatiotemporal dependencies and extracts compact, semantically meaningful representations from time-series sensor inputs in an end-to-end manner, directly informing control decisions. Its core innovation lies in decoupling perception encoding from control policy learning, enabling bandwidth-performance co-optimization at the semantic level. Evaluated on multi-instance inverted pendulum simulations, TS-JEPA achieves over 98% system stability using less than 10% of the original video bandwidth—substantially outperforming conventional compression-reconstruction baselines.
📝 Abstract
In remote control systems, transmitting large data volumes (e.g. video feeds) from wireless sensors to faraway controllers is challenging when the uplink channel capacity is limited (e.g. RedCap devices or massive wireless sensor networks). Furthermore, the controllers often only need the information-rich components of the original data. To address this, we propose a Time-Series Joint Embedding Predictive Architecture (TS-JEPA) and a semantic actor trained through self-supervised learning. This approach harnesses TS-JEPA's semantic representation power and predictive capabilities by capturing spatio-temporal correlations in the source data. We leverage this to optimize uplink channel utilization, while the semantic actor calculates control commands directly from the encoded representations, rather than from the original data. We test our model through multiple parallel instances of the well-known inverted cart-pole scenario, where the approach is validated through the maximization of stability under constrained uplink channel capacity.