Discovering Lattice Reduction Strategies via Self-Play

2026-06-13

AI Summary

This work addresses the poor basis quality produced by the LLL algorithm in high-dimensional lattices, which often fails to meet practical requirements. The authors formulate lattice reduction as a single-agent Markov decision process and propose a novel approach that integrates AlphaZero-style self-play with adaptive-horizon Monte Carlo Tree Search (MCTS) to learn improved reduction strategies directly within the original LLL operation space. Their method employs a deep residual network enhanced with an entropy-gated expansion mechanism. Remarkably, the resulting policy, DeltaStar—trained solely on 8-dimensional q-ary lattices—requires fewer row operations than LLL and demonstrates strong zero-shot generalization to lattices with unknown moduli and dimensions up to 32, substantially enhancing both reduction efficiency and generalization capability.

Abstract

The Lenstra-Lenstra-Lovász (LLL) algorithm is a seminal contribution to computer science used for lattice basis reduction, yet its polynomial-time outputs produce bases that are far from optimal as the dimension grows. We show that deep reinforcement learning can discover strictly superior, generalizable reduction strategies by interacting with the primitive action space of LLL. We formulate lattice reduction as a single-player Markov Decision Process (MDP) and train a deep residual network using an AlphaZero-style self-play pipeline augmented with adaptive-horizon MCTS (Monte Carlo Tree Search), which couples multi-step network predictions with an entropy-gated expansion mechanism. The resulting policy, DeltaStar, is trained exclusively on small $8$-dimensional $q$-ary lattices and requires fewer primitive row operations than LLL. Crucially, it generalizes zero-shot to unseen moduli and higher dimensions up to $n=32$ without retraining.

Problem

• lattice reduction
• LLL algorithm
• reinforcement learning
• generalization
• basis optimization

Innovation

• lattice reduction
• deep reinforcement learning
• self-play
• AlphaZero
• Monte Carlo Tree Search