Towards Evolving Context Parameterization for Large Language Models

2026-09-12

AI Summary

研究解决大型语言模型在持续更新情境下的信息保留与更新问题，提出无需训练的方法PLUME，通过构建全局更新表示和激活记忆证据来实现有效信息管理。

Abstract

Context parameterization enables large language models (LLMs) to internalize contexts into reusable model parameters, avoiding repeated processing across subsequent queries. However, existing methods typically assume static contexts and lack explicit mechanisms for distinguishing validity states under continual updates. To study this real-world scenario, we formalized the Memory Updating with Sequential Evolution (MUSE) task and constructed MUSE-bench to evaluate update incorporation and unaffected-information preservation. The resulting challenge requires preserving the global state while adjusting the contribution of memory evidence. Motivated by this, we proposed PLUME, a training-free method that constructs a global update representation, activates memory evidence to form a local parameter view, and adaptively integrates their predictions during decoding. Comprehensive evaluation on MUSE-bench demonstrated PLUME's effectiveness in sequential evolution settings, yielding relative improvements of 29.9% in average ROUGE-L Recall and 54.9% in LLM-as-a-Judge. Our codes are available at: https://github.com/xiaobingshi-LLM/PLUME.

Problem

• context parameterization
• large language models
• continual updates
• global state
• memory evidence

Innovation

• Memory Updating with Sequential Evolution
• PLUME
• global update representation
• adaptive integration