Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs

2026-06-11

AI Summary

Current game-playing evaluations rely on single scalar metrics, which inadequately capture the multidimensional strategic reasoning capabilities of large language models in uncertain environments. This work introduces a benchmark platform based on no-limit Texas Hold’em and proposes a three-tier memory architecture—comprising within-hand, within-session, and cross-session memory—together with a nine-dimensional decomposition framework for strategic reasoning. Through large-scale adversarial simulations and memory ablation studies, the evaluation reveals fine-grained performance characteristics. Analysis of 50,000 hands demonstrates a significant discrepancy between chip-based rankings and multidimensional capability scores, indicating that cross-dimensional consistency is a more reliable indicator of true strategic proficiency than peak performance along any single axis, thereby challenging the validity of conventional leaderboard paradigms.

Abstract

Strategic reasoning under uncertainty underpins consequential decisions in negotiation, finance, and policy, but prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined. We introduce Poker Arena, a no-limit Texas Hold'em tournament platform that couples a three-layer memory architecture (within-hand, session, and cross-session) with a nine-axis cognitive profile decomposing strategic reasoning into interpretable dimensions such as bet-sizing calibration and positional awareness. We evaluate seven frontier models across 50 sessions of 1,000 hands and a controlled memory ablation; tournament chips and aggregate axis score order the field differently: Claude Opus 4.6 wins +$15,730 chips with 14 first-place finishes, yet ranks only fifth of seven on mean axis score, while persistent memory helps some models and hurts others. These findings show that multi-axis evaluation surfaces capability structure that scalar leaderboards systematically misrank, with cross-dimensional consistency outweighing peak performance on any single axis.

Problem

• strategic reasoning
• multi-axis evaluation
• LLMs
• game-play benchmarks
• capability structure

Innovation

• multi-axis evaluation
• strategic reasoning
• memory architecture
• large language models
• game-based benchmarking