On Improving Neurosymbolic Learning by Exploiting the Representation Space

2026-02-08

AI Summary

This work addresses the challenge in neuro-symbolic learning where logical constraints induce an exponential explosion in the label combination space, hindering model training. The authors propose a pruning method grounded in instance representation similarity, formulating the search for feasible label assignments that satisfy logical formulas as an integer linear programming problem. This approach dramatically compresses the search space while strictly preserving logical consistency. Notably, it is the first to incorporate representation similarity into neuro-symbolic learning under logical constraints and integrates seamlessly with existing training algorithms. Evaluated across 16 complex benchmark tasks, the method substantially enhances the performance of state-of-the-art neuro-symbolic engines—Scallop, Dolphin, and ISED—achieving accuracy gains of up to 48%, 53%, and 8%, respectively, and establishing new state-of-the-art results.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

We study the problem of learning neural classifiers in a neurosymbolic setting where the hidden gold labels of input instances must satisfy a logical formula. Learning in this setting proceeds by first computing (a subset of) the possible combinations of labels that satisfy the formula and then computing a loss using those combinations and the classifiers'scores. One challenge is that the space of label combinations can grow exponentially, making learning difficult. We propose a technique that prunes this space by exploiting the intuition that instances with similar latent representations are likely to share the same label. While this intuition has been widely used in weakly supervised learning, its application in our setting is challenging due to label dependencies imposed by logical constraints. We formulate the pruning process as an integer linear program that discards inconsistent label combinations while respecting logical structure. Our approach, CLIPPER, is orthogonal to existing training algorithms and can be seamlessly integrated with them. Across 16 benchmarks over complex neurosymbolic tasks, we demonstrate that CLIPPER boosts the performance of state-of-the-art neurosymbolic engines like Scallop, Dolphin, and ISED by up to 48%, 53%, and 8%, leading to state-of-the-art accuracies.

Problem

• neurosymbolic learning
• logical constraints
• label combinations
• exponential space
• hidden labels

Innovation

• neurosymbolic learning
• label space pruning
• integer linear programming
• representation similarity
• logical constraints