FlashBack: Consistency Model-Accelerated Shared Autonomy

2025-05-22

AI Summary

Existing shared autonomy approaches rely on user goal priors, handcrafted reward functions, or real-time user queries; while diffusion models avoid such assumptions, their high computational cost hinders real-time deployment. This work introduces consistency models—the first such application in shared autonomy—leveraging probability-flow ODEs for single-step sampling and enabling intervention at intermediate latent states. Crucially, it achieves millisecond-latency, intensity-tunable assistance without requiring any user priors or online queries. The method unifies consistency modeling, diffusion distillation, motion planning, and closed-loop control. Evaluated across diverse simulated and real-robot tasks, it accelerates inference by over an order of magnitude and significantly outperforms state-of-the-art methods in task success rate.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Shared autonomy is an enabling technology that provides users with control authority over robots that would otherwise be difficult if not impossible to directly control. Yet, standard methods make assumptions that limit their adoption in practice-for example, prior knowledge of the user's goals or the objective (i.e., reward) function that they wish to optimize, knowledge of the user's policy, or query-level access to the user during training. Diffusion-based approaches to shared autonomy do not make such assumptions and instead only require access to demonstrations of desired behaviors, while allowing the user to maintain control authority. However, these advantages have come at the expense of high computational complexity, which has made real-time shared autonomy all but impossible. To overcome this limitation, we propose Consistency Shared Autonomy (CSA), a shared autonomy framework that employs a consistency model-based formulation of diffusion. Key to CSA is that it employs the distilled probability flow of ordinary differential equations (PF ODE) to generate high-fidelity samples in a single step. This results in inference speeds significantly than what is possible with previous diffusion-based approaches to shared autonomy, enabling real-time assistance in complex domains with only a single function evaluation. Further, by intervening on flawed actions at intermediate states of the PF ODE, CSA enables varying levels of assistance. We evaluate CSA on a variety of challenging simulated and real-world robot control problems, demonstrating significant improvements over state-of-the-art methods both in terms of task performance and computational efficiency.

Problem

• Overcoming high computational complexity in shared autonomy systems
• Enabling real-time assistance with single-step high-fidelity samples
• Providing varying assistance levels without prior user policy knowledge

Innovation

• Uses consistency model-based diffusion formulation
• Employs distilled PF ODE for single-step sampling
• Enables real-time assistance with varied intervention levels