Tackling GNARLy Problems: Graph Neural Algorithmic Reasoning Reimagined through Reinforcement Learning

2025-09-23

AI Summary

Neural Algorithmic Reasoning (NAR) suffers from inherent limitations in reconstructive reasoning, multi-solution inference, and NP-hard combinatorial optimization, heavily relying on expert-crafted algorithmic priors. To address this, we formulate algorithmic trajectory learning as a Markov Decision Process and propose the first general-purpose, end-to-end trainable framework that integrates Graph Neural Networks with Reinforcement Learning—requiring no prior algorithmic knowledge. Our method employs imitation learning for policy initialization followed by reinforcement learning fine-tuning. On the CLRS-30 benchmark for graph algorithmic tasks, it achieves near-perfect graph-level accuracy (~100%), matching or surpassing specialized NAR approaches—even on problems lacking known efficient algorithms. Key contributions include: (i) unifying algorithmic reasoning as sequential decision-making; (ii) enabling unsupervised generation of valid solutions; and (iii) ensuring strong generalization and problem-agnostic applicability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Neural Algorithmic Reasoning (NAR) is a paradigm that trains neural networks to execute classic algorithms by supervised learning. Despite its successes, important limitations remain: inability to construct valid solutions without post-processing and to reason about multiple correct ones, poor performance on combinatorial NP-hard problems, and inapplicability to problems for which strong algorithms are not yet known. To address these limitations, we reframe the problem of learning algorithm trajectories as a Markov Decision Process, which imposes structure on the solution construction procedure and unlocks the powerful tools of imitation and reinforcement learning (RL). We propose the GNARL framework, encompassing the methodology to translate problem formulations from NAR to RL and a learning architecture suitable for a wide range of graph-based problems. We achieve very high graph accuracy results on several CLRS-30 problems, performance matching or exceeding much narrower NAR approaches for NP-hard problems and, remarkably, applicability even when lacking an expert algorithm.

Problem

• Learning algorithm trajectories as Markov Decision Processes
• Addressing limitations of Neural Algorithmic Reasoning approaches
• Solving graph-based problems without requiring expert algorithms

Innovation

• Reframes algorithm learning as Markov Decision Process
• Proposes GNARL framework for graph-based RL translation
• Achieves high accuracy without expert algorithm supervision