Reinforcement Learning of Flexible Policies for Symbolic Instructions With Adjustable Mapping Specifications

2025-01-31

IEEE Robotics and Automation Letters

AI Summary

To address the challenge of dynamically interpreting identical symbolic instructions based on contextual state in multi-view robotic equipment inspection, this paper proposes a reinforcement learning framework that decouples symbolic semantics from mapping specifications. Methodologically, it (1) explicitly separates symbolic instruction representation—formalized in Linear Temporal Logic—from learnable mapping specifications; (2) introduces a specification-aware state modulation mechanism to enable state-dependent semantic adaptation; and (3) designs a symbol-cardinality-driven task curriculum learning strategy for progressive complexity scaling. Evaluated in a 3D simulation environment, the approach achieves a 27.4% higher success rate on flexible mapping tasks compared to a context-aware multitask RL baseline, demonstrating significantly improved generalization and environmental robustness.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Symbolic task representation is a powerful tool for encoding human instructions and domain knowledge. Such instructions guide robots to accomplish diverse objectives and meet constraints through reinforcement learning (RL). Most existing methods are based on fixed mappings from environmental states to symbols. However, in inspection tasks, where equipment conditions must be evaluated from multiple perspectives to avoid errors of oversight, robots must fulfill the same symbol from different states. To help robots respond to flexible symbol mapping, we propose representing symbols and their mapping specifications separately within an RL policy. This approach imposes on RL policy to learn combinations of symbolic instructions and mapping specifications, requiring an efficient learning framework. To cope with this issue, we introduce an approach for learning flexible policies called Symbolic Instructions with Adjustable Mapping Specifications (SIAMS). This paper represents symbolic instructions using linear temporal logic (LTL), a formal language that can be easily integrated into RL. Our method addresses the diversified completion patterns of instructions by (1) a specification-aware state modulation, which embeds differences in mapping specifications in state features, and (2) a symbol-number-based task curriculum, which gradually provides tasks according to the learning's progress. Evaluations in 3D simulations with discrete and continuous action spaces demonstrate that our method outperforms context-aware multitask RL comparisons.

Problem

• Reinforcement Learning
• Symbolic Instruction Execution
• Adaptive Task Complexity

Innovation

• SIAMS
• Reinforcement Learning
• Adaptable State Adjustment