Neural Algorithmic Reasoning with Multiple Correct Solutions

2024-09-11

arXiv.org

AI Summary

Existing neural algorithmic reasoning (NAR) methods produce only a single solution, rendering them inadequate for multi-solution problems—e.g., enumerating all shortest paths. Method: This work introduces the first NAR framework supporting *multi-solution generation*, comprising three core components: (i) task-specific multi-solution data synthesis, (ii) graph neural network modeling of algorithm execution dynamics, and (iii) stochastic sampling over the solution space followed by deterministic verification. Contribution/Results: By integrating customizable data construction with verifiable sampling, our framework consistently generates diverse, correct solution sets for classical algorithmic tasks—including Bellman–Ford and depth-first search—achieving both correctness and solution diversity. To our knowledge, this is the first systematic study in NAR addressing multi-solution reasoning. It establishes a novel paradigm for algorithmic tasks requiring exhaustive enumeration of valid solutions, advancing the expressivity and reliability of neural reasoning systems.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Neural Algorithmic Reasoning (NAR) aims to optimize classical algorithms. However, canonical implementations of NAR train neural networks to return only a single solution, even when there are multiple correct solutions to a problem, such as single-source shortest paths. For some applications, it is desirable to recover more than one correct solution. To that end, we give the first method for NAR with multiple solutions. We demonstrate our method on two classical algorithms: Bellman-Ford (BF) and Depth-First Search (DFS), favouring deeper insight into two algorithms over a broader survey of algorithms. This method involves generating appropriate training data as well as sampling and validating solutions from model output. Each step of our method, which can serve as a framework for neural algorithmic reasoning beyond the tasks presented in this paper, might be of independent interest to the field and our results represent the first attempt at this task in the NAR literature.

Problem

• Neural Algorithmic Reasoning
• Multi-Solution Problems
• Shortest Path

Innovation

• Multi-answer NAR
• Neural Algorithm Reasoning
• Algorithmic Problem Solving