Enhancing Large Language Models with Neurosymbolic Reasoning for Multilingual Tasks

2025-06-03

AI Summary

Large language models (LLMs) face significant challenges in long-context, multi-objective reasoning—including difficulty in fact localization due to information dispersion, weak cross-document synthesis, and poor interpretability. To address these issues, this paper proposes the Neural-Symbolic Augmented Reasoning (NSAR) framework. NSAR innovatively introduces dynamic symbolic fact extraction and real-time executable Python code generation *during inference*, establishing a closed-loop synergy between neural language modeling and symbolic logic. The framework integrates multilingual retrieval-augmented generation (RAG), symbolic fact extraction, and program synthesis, supporting seven languages and variable-length contexts. Experimental results demonstrate that NSAR substantially outperforms state-of-the-art RAG systems and advanced prompting methods across multilingual and variable-length long-text reasoning benchmarks, achieving significant gains in accuracy and robustness. Crucially, NSAR delivers both strong reasoning capability and end-to-end process interpretability through transparent, executable symbolic reasoning traces.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Neuro-Symbolic Learning
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) often struggle to perform multi-target reasoning in long-context scenarios where relevant information is scattered across extensive documents. To address this challenge, we introduce NeuroSymbolic Augmented Reasoning (NSAR), which combines the benefits of neural and symbolic reasoning during inference. NSAR explicitly extracts symbolic facts from text and generates executable Python code to handle complex reasoning steps. Through extensive experiments across seven languages and diverse context lengths, we demonstrate that NSAR significantly outperforms both a vanilla RAG baseline and advanced prompting strategies in accurately identifying and synthesizing multiple pieces of information. Our results highlight the effectiveness of combining explicit symbolic operations with neural inference for robust, interpretable, and scalable reasoning in multilingual settings.

Problem

• Enhancing multilingual reasoning in scattered long-context scenarios
• Combining neural and symbolic reasoning for accurate information synthesis
• Improving interpretability and scalability in multi-target reasoning tasks

Innovation

• Combines neural and symbolic reasoning for inference
• Extracts symbolic facts and generates Python code
• Outperforms RAG and prompting in multilingual tasks