🤖 AI Summary
Existing differentiable forward-chaining inference methods suffer from prohibitive memory overhead, hindering the exploitation of first-order logic’s expressive power and failing to support cross-scenario analogical reasoning in abstract visual reasoning.
Method: We propose NEUMANN, a memory-efficient, graph-structured differentiable forward reasoner that enables functorial execution of structured programs. We formally introduce the novel task of “behind-the-scenes visual reasoning” and design an interpretable, efficient program induction algorithm. Our approach integrates neural-symbolic computation, differentiable logic programming, graph message passing, and structural learning.
Contribution/Results: On standard abstract visual reasoning benchmarks and the new behind-the-scenes reasoning task, NEUMANN significantly outperforms pure neural, pure symbolic, and state-of-the-art neurosymbolic baselines. It uniquely combines logical interpretability with end-to-end gradient-based optimization, enabling scalable and principled visual reasoning.
📝 Abstract
Visual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.
