First Make It Playable, Then Make It Good: Staged Interaction Learning for Small Dialogue-Game Agents

2026-08-27

AI Summary

研究通过三步微调方法提升小型对话游戏代理性能，主要解决游戏可玩性和决策质量，最终模型在Playpen验证中取得高分。

Abstract

We present Qwen-GuidePlay-2B, a 2B-parameter language model for dialogue-game interaction. We fine-tune Qwen3.5-2B using three steps: a) SFT on only successful game trajectories from Playpen, b) weighted turn-level SFT, and c) teacher-guided SFT. The teacher model (which is a larger model) is only used to fix formatting and evaluate examples, but does not create new gold actions. Our final model scores 57.12 clemscore and 42.68 statscore on the public Playpen validation. In the officially released challenge results, our model obtains the second-highest Playpen clemscore delta among submitted systems (which is approximately +36 over its base model). Our findings suggest that imitating full trajectories helps with playability, while turn-level and teacher-guided training usually improve decision-making and increase the overall score. Alternative procedurally heavy approaches like replay-repair and hard-example mining did not help, which suggests that small models are performant simply by using careful curation strategies rather than aggressive changes. We make available both the model and the code for reproducibility.

Problem

• dialogue-game interaction
• playability
• decision-making
• small models

Innovation

• Qwen-GuidePlay-2B
• staged interaction learning
• supervised fine-tuning (SFT)
• teacher-guided SFT
• playability and decision-making