Enhancing Player Enjoyment with a Two-Tier DRL and LLM-Based Agent System for Fighting Games

2025-04-10

AI Summary

To address the challenge of balancing skill proficiency and player enjoyment in fighting game AI, this paper proposes a two-tiered adversarial agent framework integrating deep reinforcement learning (DRL) and large language models (LLMs). At the lower tier, a hybrid PPO/SAC training paradigm with modular reward shaping generates high-skill, stylistically diverse DRL agents. At the upper tier, an LLM serves as a meta-agent that dynamically models player behavior and real-time feedback to select and schedule optimal opponent strategies. This work is the first to explicitly model and incorporate player enjoyment into the core architecture of fighting game AI, enabling joint optimization of skill execution and stylistic diversity. Experiments demonstrate a 64.36%–156.36% improvement in high-skill move execution rates and statistically significant stylistic differentiation among agents. A small-scale user study confirms a significant increase in overall player enjoyment (p < 0.01).

Technology Category

• Multiagent Systems: Adversarial Agents
• Game Theory and Economic Paradigms: Adversarial Learning
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Deep reinforcement learning (DRL) has effectively enhanced gameplay experiences and game design across various game genres. However, few studies on fighting game agents have focused explicitly on enhancing player enjoyment, a critical factor for both developers and players. To address this gap and establish a practical baseline for designing enjoyability-focused agents, we propose a two-tier agent (TTA) system and conducted experiments in the classic fighting game Street Fighter II. The first tier of TTA employs a task-oriented network architecture, modularized reward functions, and hybrid training to produce diverse and skilled DRL agents. In the second tier of TTA, a Large Language Model Hyper-Agent, leveraging players' playing data and feedback, dynamically selects suitable DRL opponents. In addition, we investigate and model several key factors that affect the enjoyability of the opponent. The experiments demonstrate improvements from 64. 36% to 156. 36% in the execution of advanced skills over baseline methods. The trained agents also exhibit distinct game-playing styles. Additionally, we conducted a small-scale user study, and the overall enjoyment in the player's feedback validates the effectiveness of our TTA system.

Problem

• Enhancing player enjoyment in fighting games using DRL and LLM
• Designing diverse skilled agents with modular rewards and hybrid training
• Dynamic opponent selection based on player data and feedback

Innovation

• Two-tier DRL and LLM-based agent system
• Modular reward functions and hybrid training
• Dynamic opponent selection using player data