Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

2026-09-15

AI Summary

为解决大型语言模型持续适应变化任务和环境的挑战，提出动态检索策略生成框架DRPG，通过历史数据和环境反馈生成特定任务策略以不断改进模型。

Abstract

Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remains a key challenge. Existing memory-augmented approaches retrieve individual past examples as direct references, but do not explicitly synthesize actionable strategies from them, causing the same types of errors to recur. We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement. We evaluate DRPG across six benchmarks spanning text-to-SQL, question answering, medical diagnosis, and Python programming, using seven LLMs from both proprietary and open-weight families. DRPG outperforms strong baselines across most datasets and models. Further analysis demonstrates that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators. We also find that the benefit of policy-level guidance depends on task characteristics, offering practical insights into when and under what conditions this mechanism is most effective.

Problem

• Continual Adaptation
• Large Language Models
• Memory-Augmented Approaches
• Evolving Tasks

Innovation

• Dynamic Retrieval-based Policy Generation (DRPG)
• continual LLM improvement
• memory-augmented approaches
• environment feedback
• task-specific policies