Lattice Deduction Transformers

2026-05-08

AI Summary

Current large language models fail on symbolic logical reasoning tasks such as Sudoku and maze solving. This work proposes a novel architecture that integrates lattice theory with a recursive Transformer, approximating logical deduction by projecting hidden states onto lattices between forward passes. It introduces domain-agnostic supervision signals derived from abstract interpretation to emulate the search-based reasoning of constraint solvers within policy-based training. The approach achieves, for the first time, empirically reliable and verifiable reasoning: the model either produces a correct solution or explicitly abstains. With only 0.8 million parameters, it attains 100% accuracy on Sudoku-Extreme and Snowflake Sudoku; with 1.8 million parameters, it reaches 99.9% accuracy on Maze-Hard—tasks on which mainstream large language models achieve 0% accuracy.

Abstract

We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic, abstract-interpretation-based approximation of the set of solution candidates. An $800$K-parameter LDT achieves $100\%$ accuracy on Sudoku-Extreme and Snowflake Sudoku, at a fraction of the training cost of prior small recurrent reasoners, while remaining empirically sound: the model returns a correct answer or abstains. A $1.8$M-parameter variant reaches $99.9\%$ accuracy on Maze-Hard. Frontier LLMs score $0\%$ on all three benchmarks.

Problem

• logical reasoning
• constraint solving
• neural deduction
• Sudoku
• maze solving

Innovation

• Lattice Deduction Transformer
• abstract interpretation
• sound reasoning
• recurrent transformer
• constraint solving