Chessformer: A Unified Architecture for Chess Modeling

2026-05-18

AI Summary

Traditional chess models struggle to simultaneously achieve strong playing strength, accurate prediction of human moves, and interpretability, often overlooking the geometric structure of the chessboard. This work proposes Chessformer, an encoder-based Transformer architecture that treats board squares as tokens and incorporates a geometric attention bias (GAB) to dynamically encode positional information. It further introduces an attention-driven source–target policy head. For the first time, this unified architecture jointly optimizes all three objectives: it attains 57.1% accuracy in human move prediction with fewer than one-quarter the parameters of the previous state-of-the-art model, gains over 100 Elo when integrated into Leela Chess Zero—defeating Stockfish in practical play—and enables fine-grained interpretability through its attention mechanisms.

Abstract

Chess has long served as a canonical testbed for artificial intelligence, but modeling approaches for its central tasks have diverged. Maximizing playing strength, predicting human play, and enabling interpretability are typically solved with disparate architectures, and these designs are often misaligned with the geometry of the domain. This raises the natural question of whether these objectives require separate modeling paradigms, or if there exists a single architecture that supports them simultaneously. We introduce Chessformer, a unified architecture that advances the state of the art on all three central goals in chess modeling. Chessformer is an encoder-only transformer that represents board squares as tokens, augments self-attention with a novel dynamic positional encoding called Geometric Attention Bias (GAB) that adapts to domain-specific geometry, and predicts actions with an attention-based source-destination policy head. We evaluate Chessformer on each front. First, we develop \maiathree, a family of models for human move prediction that reaches 57.1\% move-matching accuracy, significantly surpassing the previous state of the art with fewer than a quarter of the parameters. Second, we integrate Chessformer into Leela Chess Zero, a leading open-source engine, adding over 100 Elo of playing strength and resulting in tournament victories over Stockfish in major computer chess competitions. Third, we show that Chessformer's square-token design makes attention patterns and activations directly attributable to board squares, enabling granular interpretability analyses that prior architectures do not naturally support. More broadly, our results demonstrate that aligning a model's tokenization, positional encoding, and output design with the underlying structure of a domain can yield simultaneous gains in performance, human compatibility, and interpretability.

Problem

• chess modeling
• unified architecture
• human move prediction
• playing strength
• interpretability

Innovation

• Chessformer
• Geometric Attention Bias
• unified architecture
• square-token representation
• attention-based policy head