Knowledge Vector of Logical Reasoning in Large Language Models

2026-04-26

AI Summary

This study investigates the knowledge representations of deductive, inductive, and abductive reasoning in large language models and their interrelationships. By modeling each reasoning type as a distinct knowledge vector within a linear representation space, the work reveals—for the first time—their near-orthogonal representational structure. Building on insights from cognitive science, the authors propose a complementary optimization framework that integrates a complementarity loss with subspace constraints to jointly enhance each reasoning capability. Experimental results demonstrate that the optimized reasoning vectors consistently improve performance across multiple tasks, while also uncovering both shared mechanisms and modality-specific characteristics underlying the three forms of logical inference.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Logical reasoning serve as a central capability in LLMs and includes three main forms: deductive, inductive, and abductive reasoning. In this work, we study the knowledge representations of these reasoning types in LLMs and analyze the correlations among them. Our analysis shows that each form of logical reasoning can be captured as a reasoning-specific knowledge vector in a linear representation space, yet these vectors are largely independent of each other. Motivated by cognitive science theory that these subforms of logical reasoning interact closely in the human brain, as well as our observation that the reasoning process for one type can benefit from the reasoning chain produced by another, we further propose to refine the knowledge representations of each reasoning type in LLMs to encourage complementarity between them. To this end, we design a complementary subspace-constrained refinement framework, which introduces a complementary loss that enables each reasoning vector to leverage auxiliary knowledge from the others, and a subspace constraint loss that prevents erasure of their unique characteristics. Through steering experiments along reasoning vectors, we find that refined vectors incorporating complementary knowledge yield consistent performance gains. We also conduct a mechanism-interpretability analysis of each reasoning vector, revealing insights into the shared and specific features of different reasoning in LLMs.

Problem

• logical reasoning
• knowledge representation
• large language models
• reasoning complementarity
• deductive-inductive-abductive reasoning

Innovation

• knowledge vector
• logical reasoning
• complementary subspace
• large language models
• reasoning representation