Jointly Learning Predicates and Actions Enables Zero-Shot Skill Composition

2026-05-19

AI Summary

Existing learning-from-demonstration approaches struggle to generalize to novel compositions of known skills without retraining and lack explicit modeling of the relationship between actions and symbolic outcomes. This work proposes the Predicate-Action Skill (PACTS) framework, which, for the first time, jointly models action trajectories and predicate belief trajectories as a unified generative process, endowing the policy with intrinsic symbolic reasoning capabilities. By integrating a closed-loop visuomotor policy, a joint generative model, and predicate belief estimation, PACTS ensures consistent generation of actions and their symbolic consequences and enables zero-shot compositional skill planning through online predicate prediction. Experiments demonstrate that PACTS not only improves both action generation and predicate classification performance but also reliably executes unseen skill combinations in complex tasks without fine-tuning.

Abstract

Learning from Demonstration (LfD) enables robots to learn complex behaviors from expert examples, yet existing approaches often fail to generalize to new compositions of known skills without retraining. Modern generative policies model distributions over action trajectories alone, thus are unable to reason about the symbolic outcomes required for robust composition. We propose that skills should jointly model action trajectories and the symbolic outcomes they induce. To address this gap, we introduce Predicate Action Skills (PACTS), a class of closed-loop visuomotor policies that model skills as a joint generative process over action and predicate belief trajectories, producing coherent action-outcome rollouts within a single model. Jointly generating actions and predicates enables PACTS to learn internal representations that improve both action generation and predicate classification. Furthermore, we demonstrate zero-shot composition of learned skills via planning by leveraging online predicate predictions from PACTS as a symbolic interface for sequencing and monitoring execution. Project website: https://planpacts.github.io/

Problem

• Learning from Demonstration
• Zero-Shot Composition
• Symbolic Reasoning
• Skill Generalization
• Action-Predicate Modeling

Innovation

• zero-shot composition
• joint generative modeling
• symbolic predicates
• visuomotor policies
• learning from demonstration