Temporal Logic Guided Universal Task Representations for Reinforcement Learning

2026-08-15

AI Summary

This study addresses the limitations of poor generalizability in existing task representations and the inefficiency of gradient-dependent updates by proposing the LOTUS framework. Leveraging Linear Temporal Logic to construct universal task representations, this method introduces a bisimulation metric to provide theoretical guarantees for behavioral equivalence and robustness while seamlessly integrating with reinforcement learning algorithms. Experimental results demonstrate that LOTUS accelerates single-task convergence by over 20% and improves zero-shot task success rates by 15%–45%. Furthermore, multi-task generalization capability is enhanced by more than 25%, significantly boosting both policy learning efficiency and generalization performance across diverse scenarios.

Abstract

Task guided agents demonstrate strong performance in a wide range of complex tasks. However, most existing task representation algorithms are tailored to specific contexts and struggle to generalize across diverse scenarios. Moreover, they typically depend on gradient signals from reinforcement learning controllers to update their weights, which can degrade both representation quality and learning efficiency. To overcome these limitations, we propose LOTUS, a temporal logic inspired universal task representation framework that can be seamlessly integrated into any RL algorithm to enhance agent performance across diverse task settings. Specifically, we design a novel task representation architecture capable of modeling relationships and extracting task semantics from LTL formulas. We further introduce a more effective update mechanism that treats the LTL encoder as a policy, thereby improving representation capacity. To enhance stability and robustness, LOTUS leverages the bisimulation metric, which provides theoretical guarantees for LTL representation, including behavioral equivalence, optimality fidelity, and trajectory robustness. Experimental results show that LOTUS outperforms most existing methods in learning efficiency, generalization capability, and representation quality. Specifically, LOTUS accelerates convergence over 20% in single-task scenarios, achieves a 15%-45% higher success rate in unseen manipulation tasks, and improves generalization performance over 25% in complex multi-task environments with increased sub-goal depth or conjunctions. The corresponding code, videos, and appendix are available at: https://lotus-website.github.io/.

Problem

• Task Representation
• Reinforcement Learning
• Generalization
• Temporal Logic
• Gradient Dependency

Innovation

• Temporal Logic
• Universal Task Representation
• Bisimulation Metric
• Reinforcement Learning
• LTL Encoder