GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning

2026-04-18

AI Summary

Traditional neuro-symbolic reinforcement learning relies on handcrafted relational concepts, limiting agent generalization and autonomy in environments with dynamically shifting semantics. This work proposes GRAIL, a framework that achieves fully autonomous construction of relational concepts for the first time. GRAIL leverages large language models to provide weakly supervised initial concept representations and refines them through online environmental interaction to align with task-specific semantics. By eliminating dependence on expert knowledge, the approach effectively mitigates challenges posed by sparse rewards and concept misalignment. Evaluated on Atari games—Kangaroo, Seaquest, and Skiing—GRAIL matches or surpasses the performance of agents using human-defined concepts, while uncovering an inherent trade-off between reward maximization and the achievement of high-level objectives.

Technology Category

• Machine Learning: Reinforcement Learning
• Multiagent Systems: Multiagent Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Neuro-symbolic Reinforcement Learning (NeSy-RL) combines symbolic reasoning with gradient-based optimization to achieve interpretable and generalizable policies. Relational concepts, such as "left of" or "close by", serve as foundational building blocks that structure how agents perceive and act. However, conventional approaches require human experts to manually define these concepts, limiting adaptability since concept semantics vary across environments. We propose GRAIL (Grounding Relational Agents through Interactive Learning), a framework that autonomously grounds relational concepts through environmental interaction. GRAIL leverages large language models (LLMs) to provide generic concept representations as weak supervision, then refines them to capture environment-specific semantics. This approach addresses both sparse reward signals and concept misalignment prevalent in underdetermined environments. Experiments on the Atari games Kangaroo, Seaquest, and Skiing demonstrate that GRAIL matches or outperforms agents with manually crafted concepts in simplified settings, and reveals informative trade-offs between reward maximization and high-level goal completion in the full environment.

Problem

• Neuro-symbolic Reinforcement Learning
• relational concepts
• concept grounding
• environment-specific semantics
• manual concept definition

Innovation

• Neuro-Symbolic Reinforcement Learning
• Concept Grounding
• Large Language Models
• Relational Concepts
• Autonomous Learning