LLMs as a synthesis between symbolic and continuous approaches to language

2025-02-17

AI Summary

This work addresses the longstanding tension between symbolic and connectionist paradigms in language and cognition research, investigating whether large language models (LLMs) instantiate a novel integrative paradigm. Using mechanistic interpretability analysis, neuron activation probing, and geometric modeling of representation spaces, we systematically demonstrate that LLMs simultaneously support both discrete symbolic and distributed continuous representations: morphosyntactic knowledge emerges in near-discrete form, whereas semantic representations exhibit continuous, graded structure. Crucially, we provide the first empirical evidence that LLMs are intrinsically dynamic hybrids of symbolic and connectionist frameworks, capable of adaptively switching between representational modes during linguistic processing. These findings establish a foundational theoretical basis and methodological pathway toward a unified computational framework for language cognition, reconciling classical symbolic formalism with modern neural representationalism.

Technology Category

• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Since the middle of the 20th century, a fierce battle is being fought between symbolic and continuous approaches to language and cognition. The success of deep learning models, and LLMs in particular, has been alternatively taken as showing that the continuous camp has won, or dismissed as an irrelevant engineering development. However, in this position paper I argue that deep learning models for language actually represent a synthesis between the two traditions. This is because 1) deep learning architectures allow for both continuous/distributed and symbolic/discrete-like representations and computations; 2) models trained on language make use this flexibility. In particular, I review recent research in mechanistic interpretability that showcases how a substantial part of morphosyntactic knowledge is encoded in a near-discrete fashion in LLMs. This line of research suggests that different behaviors arise in an emergent fashion, and models flexibly alternate between the two modes (and everything in between) as needed. This is possibly one of the main reasons for their wild success; and it is also what makes them particularly interesting for the study of language and cognition. Is it time for peace?

Problem

• Synthesis of symbolic and continuous language approaches
• LLMs demonstrate dual representation capabilities
• Mechanistic interpretability in LLMs reveals near-discrete knowledge encoding

Innovation

• LLMs blend symbolic and continuous methods
• Deep learning enables dual representation modes
• Models flexibly switch between computational approaches