LLM+AL: Bridging Large Language Models and Action Languages for Complex Reasoning about Actions

2025-01-01

AI Summary

Large language models (LLMs) exhibit systematic deficiencies in search-based reasoning for complex action tasks and struggle to integrate natural-language understanding with formal action reasoning. Method: This paper proposes the first end-to-end framework that synergistically couples LLMs with action languages—specifically Answer Set Programming (ASP)—to jointly leverage LLMs’ semantic parsing and commonsense generation capabilities and ASP’s symbolic, automated reasoning for action programs. Contribution/Results: The framework enables fully automatic generation of executable action programs. It significantly outperforms state-of-the-art closed-source models—including GPT-4, Claude 3, and Gemini Ultra—across multiple complex action-reasoning benchmarks. Crucially, it achieves stable convergence with minimal human correction, whereas purely LLM-based approaches—even with iterative feedback—fail to guarantee correctness. This work establishes the first closed-loop integration of natural-language understanding and formal action reasoning, introducing a novel paradigm for verifiable and interpretable action intelligence.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have made significant strides in various intelligent tasks but still struggle with complex action reasoning tasks that require systematic search. To address this limitation, we propose a method that bridges the natural language understanding capabilities of LLMs with the symbolic reasoning strengths of action languages. Our approach, termed"LLM+AL,"leverages the LLM's strengths in semantic parsing and commonsense knowledge generation alongside the action language's proficiency in automated reasoning based on encoded knowledge. We compare LLM+AL against state-of-the-art LLMs, including ChatGPT-4, Claude 3 Opus, Gemini Ultra 1.0, and o1-preview, using benchmarks for complex reasoning about actions. Our findings indicate that, although all methods exhibit errors, LLM+AL, with relatively minimal human corrections, consistently leads to correct answers, whereas standalone LLMs fail to improve even with human feedback. LLM+AL also contributes to automated generation of action languages.

Problem

• Complex Action Reasoning
• Large Language Models
• Symbolic Reasoning Limitations

Innovation

• LLM+AL
• Large Language Models
• Action Languages