Logic-of-Thought: Empowering Large Language Models with Logic Programs for Solving Puzzles in Natural Language

2025-05-22

AI Summary

Large language models (LLMs) suffer from imprecise reasoning and insufficient search when solving complex natural language puzzles—e.g., grid-based inference and dynamic action planning. To address this, we propose Logic-of-Thought (Logot), the first framework enabling end-to-end automatic translation from LLM-generated natural language descriptions to Answer Set Programming (ASP) programs, coupled with a neuro-symbolic collaborative reasoning architecture. Logot tightly integrates LLMs’ semantic comprehension with ASP’s formal modeling and exact logical inference capabilities. Evaluated on diverse natural language puzzle benchmarks, Logot achieves near-perfect (≈100%) accuracy, substantially outperforming pure-LLM baselines. By unifying neural and symbolic AI paradigms, Logot bridges the neuro-symbolic gap and establishes a new paradigm for verifiable, interpretable hybrid reasoning. The implementation, including code and datasets, is publicly released.

Technology Category

• Knowledge Representation and Reasoning: Logic Programming
• Machine Learning: Neuro-Symbolic Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Solving puzzles in natural language poses a long-standing challenge in AI. While large language models (LLMs) have recently shown impressive capabilities in a variety of tasks, they continue to struggle with complex puzzles that demand precise reasoning and exhaustive search. In this paper, we propose Logic-of-Thought (Logot), a novel framework that bridges LLMs with logic programming to address this problem. Our method leverages LLMs to translate puzzle rules and states into answer set programs (ASPs), the solution of which are then accurately and efficiently inferred by an ASP interpreter. This hybrid approach combines the natural language understanding of LLMs with the precise reasoning capabilities of logic programs. We evaluate our method on various grid puzzles and dynamic puzzles involving actions, demonstrating near-perfect accuracy across all tasks. Our code and data are available at: https://github.com/naiqili/Logic-of-Thought.

Problem

• Enhancing LLMs with logic programs for puzzle-solving
• Bridging natural language understanding with precise reasoning
• Translating puzzles into answer set programs for accuracy

Innovation

• Combines LLMs with logic programming
• Translates puzzles into answer set programs
• Uses ASP interpreter for precise reasoning