NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Models

2025-10-10

AI Summary

Symbolic cognitive architectures like SOAR suffer from poor scalability due to reliance on manually encoded production rules; existing LLM-based rule generation studies remain largely conceptual, lacking empirical validation. Method: We propose the first natural-language-to-symbolic-rules framework for SOAR, integrating a generator-critic mechanism, retrieval-augmented self-evolving knowledge base, and environment-feedback-driven iterative optimization. Technically, it unifies Gemini/Qwen LLMs, RAG, SOAR’s symbolic execution engine, and NL2Code translation. Contribution/Results: Evaluated on the Water Jug Problem dataset, our framework achieves >86% rule generation success rate. Generated heuristic rules reduce decision cycles to 1.98× the optimal solution—just 0.1% of baseline methods’ cost—and enable small models to outperform large ones. This work provides the first experimental evidence that LLMs can reliably generate high-quality, executable symbolic rules for cognitive architectures.

Technology Category

• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Machine Learning: Neuro-Symbolic Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

SOAR, a classic symbol-based cognitive architecture, has been fostering the development of general, human-like intelligent agents. Nevertheless, its practical adoption is hindered by the laborious manual rule coding. Emerging Large Language Models (LLMs) present the immense potential for efficient rules generation. However, there is a critical gap that current research predominantly focuses on conceptual frameworks and lacks robust experimental validation. To bridge this gap, we propose extit{N}atural extit{L}anguage to extit{Gen}erative extit{Sym}bolic Rules (NL2GenSym), a novel framework that integrates LLMs with SOAR to autonomously produce generative symbolic rules from natural language. Specifically, our framework introduces a novel Execution-Grounded Generator-Critic mechanism. The LLM-based Generator, guided by a Retrieval-Augmented Generation-accessed self-evolving domain knowledge base, proposes rules from natural language. Subsequently, these rules are immediately executed within the SOAR environment to rigorously validate their correctness. Based on this execution-grounded feedback, a reflective LLM-based Critic drives the iterative refinement of these rules. Experiments on our specialized Water Jug Problem (WJP) dataset, utilizing both Gemini and Qwen series models, validate the efficacy of our framework. It achieves a success rate over 86% in generating rules from natural language. Crucially, the framework also generates novel heuristic rules, reducing average decision cycles for solving the WJP to 1.98 times the optimal solution and 1/1000 of baseline methods. Additionally, our initial experiments show that NL2GenSym enables smaller-parameter models to achieve better performance than larger counterparts.

Problem

• Automates manual rule coding for SOAR cognitive architecture using LLMs
• Generates symbolic rules from natural language with execution validation
• Improves decision efficiency and enables smaller models' competitive performance

Innovation

• Generates SOAR rules from natural language using LLMs
• Uses execution-grounded generator-critic mechanism for refinement
• Integrates retrieval-augmented knowledge base for domain guidance