Discovering a Shared Logical Subspace: Steering LLM Logical Reasoning via Alignment of Natural-Language and Symbolic Views

2026-04-21

AI Summary

This work addresses the limited performance of large language models (LLMs) in multi-step logical reasoning tasks, where effectively integrating the strengths of natural language and symbolic reasoning remains challenging. The study proposes and empirically validates, for the first time, the existence of a cross-modal aligned shared logical subspace within LLMs that unifies reasoning representations across natural and symbolic languages. Leveraging canonical correlation analysis (CCA), the authors extract this low-dimensional, highly correlated subspace from residual activations of two types of reasoning chains and develop a training-free inference steering mechanism based on it. The proposed approach achieves up to an 11-percentage-point improvement in accuracy across four logical reasoning benchmarks and demonstrates strong out-of-distribution generalization capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) still struggle with multi-step logical reasoning. Existing approaches either purely refine the reasoning chain in natural language form or attach a symbolic solver as an external module. In this work, we instead ask whether LLMs contain a shared internal logical subspace that simultaneously aligns natural-language and symbolic-language views of the reasoning process. Our hypothesis is that this logical subspace captures logical reasoning capabilities in LLMs that are shared across views while remaining independent of surface forms. To verify this, we employ Canonical Correlation Analysis on the paired residual activations from natural-language and symbolic-language reasoning chains, learning a low-dimensional subspace with maximum cross-view correlation. Furthermore, we design a training-free approach that steers LLMs reasoning chain along this logical subspace, thereby leveraging the complementary reasoning signals from both views. Experiments on four logical reasoning benchmarks demonstrate the effectiveness of our approach, improving accuracy by up to 11 percentage points and generalizing well on out-of-domain problems.

Problem

• logical reasoning
• large language models
• natural-language
• symbolic reasoning
• multi-step reasoning

Innovation

• logical subspace
• canonical correlation analysis
• natural-language reasoning
• symbolic reasoning
• training-free steering