Self-Adapting Language Models

2025-06-12

AI Summary

Current large language models (LLMs) lack the capability to autonomously update their parameters to adapt to new tasks or incorporate novel knowledge. Method: This paper introduces the first purely generative, end-to-end self-adaptation training paradigm, wherein the model autonomously generates editing instructions and synthesizes fine-tuning data to drive persistent parameter updates—without external modules or human intervention. The approach integrates supervised fine-tuning (SFT), reinforcement learning with reward derived from post-update performance, tool invocation, and self-supervised data augmentation. Contribution/Results: Our method achieves significant improvements in knowledge injection and few-shot generalization, demonstrating robust adaptation efficacy. Crucially, it provides the first empirical validation that LLMs possess an intrinsic capacity for self-directed, continual evolution—establishing a foundational step toward autonomous model adaptation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives. Given a new input, the model produces a self-edit-a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates. Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation. To train the model to produce effective self-edits, we use a reinforcement learning loop with the downstream performance of the updated model as the reward signal. Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model's own generation to control its adaptation process. Experiments on knowledge incorporation and few-shot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation. Our website and code is available at https://jyopari.github.io/posts/seal.

Problem

• LLMs lack dynamic weight adaptation mechanisms
• SEAL enables self-adaptation via self-generated data
• Improves knowledge incorporation and few-shot generalization

Innovation

• Generates self-edit data for adaptation
• Uses reinforcement learning for optimization
• Directly controls adaptation via model generation