CTSketch: Compositional Tensor Sketching for Scalable Neurosymbolic Learning

2025-03-31

AI Summary

To address the scalability limitations of neural-symbolic learning under large-scale inputs in end-to-end supervised settings, this paper introduces compositional tensor sketching: a method that recursively decomposes symbolic programs into subprograms and represents their behaviors via low-dimensional tensor sketches, enabling joint optimization of neural networks and symbolic logic. We theoretically derive an upper bound on the sketch approximation error and support end-to-end training via gradient backpropagation combined with symbolic execution. Our approach is the first to overcome performance bottlenecks beyond thousand-scale inputs, maintaining high accuracy and training stability across multiple neural-symbolic benchmark tasks. It significantly increases model capacity limits and improves training efficiency, establishing a scalable new paradigm for large-scale end-to-end neural-symbolic learning.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Many computational tasks benefit from being formulated as the composition of neural networks followed by a discrete symbolic program. The goal of neurosymbolic learning is to train the neural networks using only end-to-end input-output labels of the composite. We introduce CTSketch, a novel, scalable neurosymbolic learning algorithm. CTSketch uses two techniques to improve the scalability of neurosymbolic inference: decompose the symbolic program into sub-programs and summarize each sub-program with a sketched tensor. This strategy allows us to approximate the output distribution of the program with simple tensor operations over the input distributions and summaries. We provide theoretical insight into the maximum error of the approximation. Furthermore, we evaluate CTSketch on many benchmarks from the neurosymbolic literature, including some designed for evaluating scalability. Our results show that CTSketch pushes neurosymbolic learning to new scales that have previously been unattainable by obtaining high accuracy on tasks involving over one thousand inputs.

Problem

• Scalable neurosymbolic learning for neural-symbolic composition tasks
• Approximating symbolic program outputs via tensor sketching
• Enhancing scalability to over one thousand input tasks

Innovation

• Decomposes symbolic programs into sub-programs
• Summarizes sub-programs with sketched tensors
• Approximates output via tensor operations