PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

2026-04-01

AI Summary

This work addresses a critical limitation in current AI-powered psychological counseling systems, which predominantly rely on fine-tuning static datasets and lack the capacity for continuous, expert-like skill evolution through ongoing practice. To bridge this gap, the study introduces lifelong learning into AI counseling for the first time, proposing an experience-driven, self-evolving agent architecture. This agent integrates three core mechanisms: memory-augmented planning, skill evolution, and rejection-sampling-based reinforcement internalization, enabling cross-session continuity and autonomous model refinement. By distilling and incorporating professional competencies from real-world counseling trajectories, the proposed approach significantly outperforms strong baselines such as GPT-5.4 and Gemini-3 across multiple evaluation metrics, demonstrably enhancing response consistency and overall quality in multi-turn therapeutic dialogues.

Abstract

Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (\texttt{PsychAgent}) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.

Problem

• lifelong learning
• psychological counseling
• experience-driven
• self-evolution
• multi-session interaction

Innovation

• Lifelong Learning
• Memory-Augmented Planning
• Skill Evolution
• Rejection Fine-Tuning
• Self-Evolving Agent