PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents

2025-09-22

AI Summary

Contemporary conversational agents face three key limitations in proactive strategy planning: narrow strategy coverage, preference bias, and reliance on costly fine-tuning. To address these, we propose a fine-tuning-free synthetic strategy memory mechanism. It leverages large language models to generate a reusable strategy library via offline self-play, enabling dynamic retrieval and execution of strategic guidance during inference for cross-domain, long-horizon planning. By eliminating the need for human annotation and model training, our approach significantly improves strategy diversity and robustness. Empirical evaluation on emotion-support and persuasion dialogue tasks demonstrates superior performance over strong baselines; notably, gains remain stable across longer dialogue turns and more diverse scenarios, validating both generalization capability and practical utility.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Dialogue agents based on large language models (LLMs) have shown promising performance in proactive dialogue, which requires effective strategy planning. However, existing approaches to strategy planning for proactive dialogue face several limitations: limited strategy coverage, preference bias in planning, and reliance on costly additional training. To address these, we propose PRINCIPLES: a synthetic strategy memory for proactive dialogue agents. PRINCIPLES is derived through offline self-play simulations and serves as reusable knowledge that guides strategy planning during inference, eliminating the need for additional training and data annotation. We evaluate PRINCIPLES in both emotional support and persuasion domains, demonstrating consistent improvements over strong baselines. Furthermore, PRINCIPLES maintains its robustness across extended and more diverse evaluation settings. See our project page at https://huggingface.co/spaces/kimnamssya/Principles.

Problem

• Addresses limited strategy coverage in proactive dialogue planning
• Overcomes preference bias and costly training requirements
• Enhances proactive dialogue agents without additional training

Innovation

• Uses synthetic strategy memory from self-play simulations
• Provides reusable knowledge for strategy planning
• Eliminates need for additional training and annotation