The Speedup Paradox: Rethinking Inference Speed-Quality Trade-off in Embodied Tasks

2026-06-26

AI Summary

This work addresses a critical oversight in existing efficient inference methods for embodied intelligence: the neglect of closed-loop interaction effects on task-level performance, which leads to misguided trade-offs between acceleration and quality. The authors propose TISED, a novel analytical framework that systematically uncovers a non-monotonic relationship between inference acceleration and task performance. Specifically, in static tasks, reducing per-step latency may paradoxically increase total task completion time, whereas in dynamic tasks, judicious application of lossy optimizations—such as quantization, pruning, or asynchronous inference—can enhance success rates beyond those of unoptimized baselines. Empirical results further demonstrate that the optimal acceleration strategy is highly hardware-dependent, underscoring the necessity of task-aware inference optimization.

Abstract

Embodied foundation models have recently been widely used to improve robot generalization and task success rates. Previous works apply lossy efficient-inference techniques such as quantization, pruning, and asynchronous inference, accepting small action quality degradation in exchange for lower per-step computation cost and inter-action latency. However, unlike traditional static ML tasks, embodied tasks involve repeated interaction with the environment, and task-level performance is determined not only by per-step cost, but also by closed-loop effects unique to embodied execution, which remain insufficiently characterized in current efficient-inference studies. In this work, we propose TISED (\underline{T}ask-level \underline{I}nference \underline{S}peedup \underline{E}ffect \underline{D}ecomposition), an analytical framework that unifies diverse lossy inference optimization techniques and decomposes their effects on static and dynamic tasks, and uncovers some paradoxical effects on task-level performance: (1) on \textit{static tasks}, optimization sometimes can lengthen end-to-end per-task completion time even as per-step latency drops; (2) on \textit{dynamic tasks}, moderate lossy optimization can raise task success rate even above the baseline; and (3) the monotonicity and sweet-spot location of both effects can shift with hardware configuration. Together, our findings provide a new perspective on adapting inference optimization techniques to embodied tasks.

Problem

• embodied tasks
• inference speed-quality trade-off
• task-level performance
• efficient inference
• closed-loop effects

Innovation

• embodied intelligence
• efficient inference
• speed-quality trade-off
• TISED framework
• closed-loop execution