🤖 AI Summary
Neural algorithmic reasoning models exhibit poor out-of-distribution generalization, whereas classical algorithms guarantee distribution-agnostic determinism. To bridge this gap, we propose a discrete-continuous dual-stream architecture that enforces strict alignment between neural execution traces and the target algorithm’s finite-state machine: discrete state encoding and transition supervision model state evolution; a dual-stream interaction mechanism enables tight coordination between neural computation and formal semantics; and a verifiable framework supports formal behavioral proofs for arbitrary inputs. Our approach achieves 100% test accuracy on both single-task and multi-task algorithmic reasoning benchmarks—matching oracle performance—and, for the first time, enables end-to-end provable correctness. This significantly enhances neural models’ generalization to classical algorithms, while improving interpretability and reliability through formal guarantees.
📝 Abstract
Neural algorithmic reasoning aims to capture computations with neural networks via learning the models to imitate the execution of classic algorithms. While common architectures are expressive enough to contain the correct model in the weights space, current neural reasoners are struggling to generalize well on out-of-distribution data. On the other hand, classic computations are not affected by distributional shifts as they can be described as transitions between discrete computational states. In this work, we propose to force neural reasoners to maintain the execution trajectory as a combination of finite predefined states. To achieve that, we separate discrete and continuous data flows and describe the interaction between them. Trained with supervision on the algorithm's state transitions, such models are able to perfectly align with the original algorithm. To show this, we evaluate our approach on multiple algorithmic problems and get perfect test scores both in single-task and multitask setups. Moreover, the proposed architectural choice allows us to prove the correctness of the learned algorithms for any test~data.