PokerBench: Training Large Language Models to become Professional Poker Players

2025-01-14

AI Summary

This work addresses the evaluation and enhancement of higher-order intelligence in large language models (LLMs) within incomplete-information games—exemplified by professional poker. To this end, we introduce PokerBench, the first poker-specific benchmark comprising 11,000 expert-level pre-flop and flop scenarios, systematically assessing mathematical reasoning, strategic planning, and opponent modeling. We establish poker as a novel paradigm for evaluating LLMs’ higher-order cognition; propose a verifiable evaluation protocol wherein PokerBench scores exhibit strong correlation with actual win rates against human experts (r > 0.85); and identify fundamental limitations of supervised fine-tuning, advocating reinforcement learning and game-theoretic alignment as key optimization pathways. Experiments across GPT-4, Llama, and Gemma show that state-of-the-art models significantly underperform theoretical optimal strategies out-of-the-box, yet achieve substantial gains post-fine-tuning—with higher PokerBench scores predicting superior real-world win rates in human–AI matches. The dataset, code, and evaluation framework are publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Game Theory and Economic Paradigms: Adversarial Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

We introduce PokerBench - a benchmark for evaluating the poker-playing abilities of large language models (LLMs). As LLMs excel in traditional NLP tasks, their application to complex, strategic games like poker poses a new challenge. Poker, an incomplete information game, demands a multitude of skills such as mathematics, reasoning, planning, strategy, and a deep understanding of game theory and human psychology. This makes Poker the ideal next frontier for large language models. PokerBench consists of a comprehensive compilation of 11,000 most important scenarios, split between pre-flop and post-flop play, developed in collaboration with trained poker players. We evaluate prominent models including GPT-4, ChatGPT 3.5, and various Llama and Gemma series models, finding that all state-of-the-art LLMs underperform in playing optimal poker. However, after fine-tuning, these models show marked improvements. We validate PokerBench by having models with different scores compete with each other, demonstrating that higher scores on PokerBench lead to higher win rates in actual poker games. Through gameplay between our fine-tuned model and GPT-4, we also identify limitations of simple supervised fine-tuning for learning optimal playing strategy, suggesting the need for more advanced methodologies for effectively training language models to excel in games. PokerBench thus presents a unique benchmark for a quick and reliable evaluation of the poker-playing ability of LLMs as well as a comprehensive benchmark to study the progress of LLMs in complex game-playing scenarios. The dataset and code will be made available at: url{https://github.com/pokerllm/pokerbench}.

Problem

• Large Language Models
• Complex Strategy Games
• Performance Enhancement

Innovation

• PokerBench
• Complex Game Strategy
• Model Evaluation