Online Learning of Counter Categories and Ratings in PvP Games

2025-02-06

AI Summary

Existing Elo-style scalar rating systems struggle to model intransitive strategic relationships (e.g., rock-paper-scissors–style counters) prevalent in competitive games; while neural rating or counter-matrix approaches can capture such intransitivity, they rely on offline training and lack real-time adaptability. This paper introduces the first online joint learning framework grounded in Elo principles, simultaneously optimizing players’ dynamic scalar ratings and discrete counter categories—thereby unifying interpretability, real-time update capability, and intransitive relationship modeling. Our method integrates online Bayesian updating, adaptive counter-category inference, and zero-sum game–informed matchmaking. It enables sub-millisecond per-match updates. Evaluated on real-world PvP data, our approach significantly improves matchmaking fairness and achieves higher win-probability prediction accuracy than static neural baselines.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents
• Machine Learning: Online Learning & Bandits

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

In competitive games, strength ratings like Elo are widely used to quantify player skill and support matchmaking by accounting for skill disparities better than simple win rate statistics. However, scalar ratings cannot handle complex intransitive relationships, such as counter strategies seen in Rock-Paper-Scissors. To address this, recent work introduced Neural Rating Table and Neural Counter Table, which combine scalar ratings with discrete counter categories to model intransitivity. While effective, these methods rely on neural network training and cannot perform real-time updates. In this paper, we propose an online update algorithm that extends Elo principles to incorporate real-time learning of counter categories. Our method dynamically adjusts both ratings and counter relationships after each match, preserving the explainability of scalar ratings while addressing intransitivity. Experiments on zero-sum competitive games demonstrate its practicality, particularly in scenarios without complex team compositions.

Problem

• Real-time learning of counter categories
• Dynamic adjustment of player ratings
• Addressing intransitivity in competitive games

Innovation

• Online update algorithm
• Real-time counter learning
• Dynamic rating adjustment