Nonlinear Performance Degradation of Vision-Based Teleoperation under Network Latency

2026-03-06

AI Summary

This study addresses the poorly understood impact of network latency on closed-loop stability in vision-based teleoperation, particularly within perception-driven control frameworks. The authors develop a Latency-Aware Visual Teleoperation (LAVT) testbed built on ROS 2, enabling precise injection of controllable one-way delays in simulation to systematically evaluate the nonlinear effects of perceptual latency on lane-keeping performance across diverse road scenarios. Through 180 closed-loop experiments, they uncover—for the first time—a sharp stability collapse within the 150–225 ms perceptual delay range: when one-way delay exceeds 150 ms, task success rate plummets from 100% to below 50%, primarily due to phase-lag-induced oscillatory instability. The work further quantifies how additional control-channel latency accelerates system failure and establishes a reproducible benchmark for latency-induced degradation in teleoperated driving.

Technology Category

• Computer Vision: Vision for Robotics & Autonomous Driving
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Teleoperation is increasingly being adopted as a critical fallback for autonomous vehicles. However, the impact of network latency on vision-based, perception-driven control remains insufficiently studied. The present work investigates the nonlinear degradation of closed-loop stability in camera-based lane keeping under varying network delays. To conduct this study, we developed the Latency-Aware Vision Teleoperation testbed (LAVT), a research-oriented ROS 2 framework that enables precise, distributed one-way latency measurement and reproducible delay injection. Using LAVT, we performed 180 closed-loop experiments in simulation across diverse road geometries. Our findings reveal a sharp collapse in stability between 150 ms and 225 ms of one-way perception latency, where route completion rates drop from 100% to below 50% as oscillatory instability and phase-lag effects emerge. We further demonstrate that additional control-channel delay compounds these effects, significantly accelerating system failure even under constant visual latency. By combining this systematic empirical characterization with the LAVT testbed, this work provides quantitative insights into perception-driven instability and establishes a reproducible baseline for future latency-compensation and predictive control strategies. Project page, supplementary video, and code are available at https://bimilab.github.io/paper-LAVT

Problem

• network latency
• vision-based teleoperation
• performance degradation
• closed-loop stability
• perception-driven control

Innovation

• network latency
• vision-based teleoperation
• closed-loop stability
• delay injection
• perception-driven control