Self-Updatable Large Language Models by Integrating Context into Model Parameters

2024-10-01

AI Summary

Addressing the fundamental challenge that large language models (LLMs) struggle to simultaneously adapt rapidly to recent interactive experiences and retain stable long-term knowledge, this paper proposes a **parameter-intrinsic self-updating mechanism**. Without external memory modules or additional parameters, it performs context-aware knowledge distillation—guided by KL divergence—to directly internalize dynamic interaction knowledge into the original model parameters. We introduce the first parameter-free, zero-fine-tuning self-updating paradigm, leveraging automatically generated, diverse question-answer pairs for efficient context compression and knowledge consolidation. Evaluated on question answering and conversational recommendation tasks, our method substantially outperforms existing memory-augmented approaches: it achieves near-optimal accuracy while improving long-term memory retention by up to 23.6%, all without incurring any additional storage overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Despite significant advancements in large language models (LLMs), the rapid and frequent integration of small-scale experiences, such as interactions with surrounding objects, remains a substantial challenge. Two critical factors in assimilating these experiences are (1) Efficacy: the ability to accurately remember recent events; (2) Retention: the capacity to recall long-past experiences. Current methods either embed experiences within model parameters using continual learning, model editing, or knowledge distillation techniques, which often struggle with rapid updates and complex interactions, or rely on external storage to achieve long-term retention, thereby increasing storage requirements. In this paper, we propose SELF-PARAM (Self-Updatable Large Language Models with Parameter Integration). SELF-PARAM requires no extra parameters while ensuring near-optimal efficacy and long-term retention. Our method employs a training objective that minimizes the Kullback-Leibler (KL) divergence between the predictions of an original model (with access to contextual information) and a target model (without such access). By generating diverse question-answer pairs related to the knowledge and minimizing the KL divergence across this dataset, we update the target model to internalize the knowledge seamlessly within its parameters. Evaluations on question-answering and conversational recommendation tasks demonstrate that SELF-PARAM significantly outperforms existing methods, even when accounting for non-zero storage requirements. This advancement paves the way for more efficient and scalable integration of experiences in large language models by embedding knowledge directly into model parameters.

Problem

• Integrate small-scale experiences into LLMs
• Ensure efficacy and retention of experiences
• Update model parameters without extra storage

Innovation

• Self-updatable LLMs without extra parameters
• KL divergence optimization for model updating
• Internalizes knowledge via diverse QA pairs