DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors

2026-04-27

AI Summary

This work addresses the challenge that synchronous executors struggle to meet the real-time demands of physical AI in dynamic tasks due to inter-block stalls. The authors propose DiscreteRTC, the first native asynchronous executor based on discrete diffusion policies, which leverages its intrinsic iterative unmasking mechanism to naturally support action completion and real-time generation—without requiring external correction, task-specific fine-tuning, or heuristic guidance. By integrating real-time chunking (RTC) with an early-stopping strategy, the approach substantially simplifies system design and reduces latency. Experiments demonstrate that, in both simulated and real-world dynamic manipulation tasks, DiscreteRTC reduces inference computation to 0.7× that of continuous RTC baselines while achieving a 50% improvement in success rate for real-time dynamic grasping.

Abstract

Unlike chatbots, physical AI must act while the world keeps evolving. Therefore, the inter-chunk pause of synchronous executors are fatal for dynamic tasks regardless of how fast the inference is. Asynchronous execution -- thinking while acting -- is therefore a structural requirement, and real-time chunking (RTC) makes it viable by recasting chunk transitions as inpainting: freezing committed actions and consistently generating the remainder. However, RTC with flow-matching policy is structurally suboptimal: its inpainting comes from inference-time corrections rather than the base policy, yielding little pre-training benefit, specific fine-tuning, heuristic guidance, and extra computation that inflates the latency. In this work, we observe that discrete diffusion policies, which generate actions by iteratively unmasking, are natural asynchronous executors that resolve all limitations at once: they are fine-tuning free since inpainting is their native operation, while early stopping further provides adaptive guidance and reduces inference cost. We propose DiscreteRTC, which replaces external corrections with native unmasking, and show on dynamic simulated benchmarks and real-world dynamic manipulation tasks that it achieves higher success rates than continuous RTC and other baselines. In summary, DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only 0.7x computation compared with generating actions from scratch, and better at execution with 50% higher success rate in real-world dynamic pick task compared with flow-matching-based RTC. More visualizations are on https://outsider86.github.io/DiscreteRTCSite/.

Problem

• asynchronous execution
• real-time chunking
• discrete diffusion
• inpainting
• dynamic tasks

Innovation

• discrete diffusion
• asynchronous execution
• real-time chunking
• inpainting
• iterative unmasking