🤖 AI Summary
This work addresses the challenge of objectively quantifying strategic rationality in real-world games for both human players and AI agents. We propose the Game Intelligence (GI) framework, which computes rationality scores from observable behavioral data—including player moves, win/loss outcomes, opponent strength, and reference AI performance. Methodologically, the framework integrates game-theoretic modeling, large-scale game record mining (>1 billion moves), and an oracle-driven error detection mechanism to identify suboptimal actions. It introduces the first computationally tractable, empirically verifiable GI scoring system and formalizes the concept of *gamingproofness* to ensure robustness against strategic manipulation or overfitting. Evaluated on over one million professional chess games, Magnus Carlsen achieves the highest GI score among humans, while Stockfish ranks first among AI systems. These results demonstrate the framework’s validity, cross-agent comparability, and broad applicability across heterogeneous decision-makers.
📝 Abstract
In this paper, I formalize intelligence measurement in games by introducing mechanisms that assign a real number -- interpreted as an intelligence score -- to each player in a game. This score quantifies the ex-post strategic ability of the players based on empirically observable information, such as the actions of the players, the game's outcome, strength of the players, and a reference oracle machine such as a chess-playing artificial intelligence system. Specifically, I introduce two main concepts: first, the Game Intelligence (GI) mechanism, which quantifies a player's intelligence in a game by considering not only the game's outcome but also the"mistakes"made during the game according to the reference machine's intelligence. Second, I define gamingproofness, a practical and computational concept of strategyproofness. To illustrate the GI mechanism, I apply it to an extensive dataset comprising over a billion chess moves, including over a million moves made by top 20 grandmasters in history. Notably, Magnus Carlsen emerges with the highest GI score among all world championship games included in the dataset. In machine-vs-machine games, the well-known chess engine Stockfish comes out on top.