🤖 AI Summary
This work addresses two key challenges in natural language multi-step deductive reasoning: poor out-of-distribution (OOD) generalization and skewed depth distribution in existing benchmarks. To tackle these, we propose IMA-GloVe-GA—a gated-attention-enhanced iterative memory network based on RNNs—integrating GloVe embeddings with explicit logical rule modeling, and introducing the first gated attention mechanism into the DeepLogic framework. Concurrently, we construct PARARULE-Plus, a large-scale, depth-balanced dataset that substantially alleviates the scarcity of deep-reasoning instances. Experiments demonstrate that our model surpasses both DeepLogic and RNN baselines on PARARULES and CONCEPTRULES. Under OOD settings involving rule reordering, it achieves higher accuracy than RoBERTa-Large. Moreover, on deep-reasoning tasks, PARARULE-Plus yields a 12.3% absolute accuracy improvement.
📝 Abstract
Combining deep learning with symbolic logic reasoning aims to capitalize on the success of both fields and is drawing increasing attention. Inspired by DeepLogic, an end-to-end model trained to perform inference on logic programs, we introduce IMA-GloVe-GA, an iterative neural inference network for multi-step reasoning expressed in natural language. In our model, reasoning is performed using an iterative memory neural network based on RNN with a gated attention mechanism. We evaluate IMA-GloVe-GA on three datasets: PARARULES, CONCEPTRULES V1 and CONCEPTRULES V2. Experimental results show DeepLogic with gated attention can achieve higher test accuracy than DeepLogic and other RNN baseline models. Our model achieves better out-of-distribution generalisation than RoBERTa-Large when the rules have been shuffled. Furthermore, to address the issue of unbalanced distribution of reasoning depths in the current multi-step reasoning datasets, we develop PARARULE-Plus, a large dataset with more examples that require deeper reasoning steps. Experimental results show that the addition of PARARULE-Plus can increase the model's performance on examples requiring deeper reasoning depths. The source code and data are available at https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language.
