How the (Tensor-) Brain uses Embeddings and Embodiment to Encode Senses and Symbols

2024-09-19

AI Summary

This study addresses the fundamental cognitive question of how the brain integrates perception, memory, and semantic understanding. We propose the Tensor Brain (TB) computational model—a biologically inspired architecture comprising two interacting layers: a sub-symbolic, global workspace–based representation layer and a symbolic index layer encoding concepts, temporal relations, and predicates. Crucially, bidirectional tensor operations mediate dynamic coupling between these layers, unifying perceptual activation with embodied semantic modulation. Our key innovation is the use of concept embeddings as “cognitive DNA” to establish the first cross-modal symbolic–subsymbolic coupling mechanism, augmented by embodied feedback for top-down semantic guidance. Evaluated on concept generalization, multimodal fusion, and context-sensitive reasoning, TB demonstrates strong cognitive coherence and neurobiological plausibility. The model advances explainable AI by enabling a closed-loop, interpretable integration of perception, memory, and semantics—offering a novel paradigm for neurocognitively grounded artificial intelligence.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Humans and AI: Brain-Sensing and Analysis
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The Tensor Brain (TB) has been introduced as a computational model for perception and memory. This paper provides an overview of the TB model, incorporating recent developments and insights into its functionality. The TB is composed of two primary layers: the representation layer and the index layer. The representation layer serves as a model for the subsymbolic global workspace, a concept derived from consciousness research. Its state represents the cognitive brain state, capturing the dynamic interplay of sensory and cognitive processes. The index layer, in contrast, contains symbolic representations for concepts, time instances, and predicates. In a bottom-up operation, sensory input activates the representation layer, which then triggers associated symbolic labels in the index layer. Conversely, in a top-down operation, symbols in the index layer activate the representation layer, which in turn influences earlier processing layers through embodiment. This top-down mechanism underpins semantic memory, enabling the integration of abstract knowledge into perceptual and cognitive processes. A key feature of the TB is its use of concept embeddings, which function as connection weights linking the index layer to the representation layer. As a concept's ``DNA,'' these embeddings consolidate knowledge from diverse experiences, sensory modalities, and symbolic representations, providing a unified framework for learning and memory.

Problem

• Embodied Cognition
• Sensory Processing
• Memory and Information Processing

Innovation

• TensorBrain
• Concept Embedding
• Memory and Understanding Mechanism