Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

2026-06-25

AI Summary

This work addresses the challenge of flexibly guiding pretrained visuomotor policies at test time to satisfy user preferences unseen during training, without requiring retraining. The authors propose ReStruct, a method that decouples a learned policy into a high-level state-machine skeleton and a low-level frozen residual controller. User preferences—such as object-centric specifications or temporal logic constraints—are incorporated into the task structure via synchronous product operations, enabling physically feasible and logically consistent behavior redirection. ReStruct is the first approach to support a broad range of inference-time preference guidance without fine-tuning or expert intervention. Evaluated in both simulation and real-world environments, it achieves up to a 25% improvement in task success rate and preference adherence over existing methods, including vision-language-action (VLA) models.

Abstract

A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.

Problem

• inference-time steering
• robot behavior
• task-structure reconfiguration
• user preferences
• policy adaptation

Innovation

• inference-time steering
• neural automaton
• task-structure reconfiguration
• physically-aware control
• residual policy