EM-NeSy: Expectation Maximization for Neurosymbolic Learning

2026-06-12

AI Summary

This work addresses a key limitation in existing neuro-symbolic models, which typically require symbolic components to be differentiable, thereby constraining the flexibility of approximate inference. The paper formalizes neuro-symbolic learning within the Expectation-Maximization (EM) framework: the E-step computes the posterior distribution over symbolic variables via exact or approximate probabilistic inference, while the M-step updates only the neural components through gradient-based optimization. This approach constitutes the first systematic integration of the EM algorithm into neuro-symbolic learning, eliminating the need for differentiable symbolic modules and unifying end-to-end training under a coherent paradigm. Experimental results demonstrate that the method significantly enhances scalability and computational efficiency while seamlessly accommodating both exact and approximate inference strategies.

Abstract

Neurosymbolic (NeSy) models integrate neural networks and symbolic reasoning for robust and interpretable AI. State-of-the-art NeSy models require that the symbolic component is expressed in a differentiable way, often complicating the use of approximate inference. We propose EM-NeSy which casts probabilistic NeSy learning as an instance of the Expectation-Maximization (EM) algorithm. In the expectation step, we compute the posterior over the neurally predicted symbols conditioned on the label via probabilistic inference. In the maximization step, we update the neural parameters based on this posterior using gradient descent only through the neural component. This formulation unlocks the full potential of the EM algorithm for NeSy learning. It allows NeSy to extend naturally to approximate reasoning without any additional modifications or differentiability requirements of the symbolic component. Furthermore, it recovers the standard end-to-end gradient-based NeSy setting under exact inference. Our experimental results demonstrate the scalability and computational efficiency of EM-NeSy.

Problem

• Neurosymbolic learning
• differentiable symbolic reasoning
• approximate inference
• Expectation-Maximization
• probabilistic inference

Innovation

• Neurosymbolic Learning
• Expectation-Maximization
• Probabilistic Inference
• Differentiable Reasoning
• Gradient-based Learning