Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching

2024-06-10

arXiv.org

AI Summary

To address the challenge that large language models (LLMs) struggle to acquire emerging knowledge due to static pretraining, this paper proposes Self-Tuning—a self-supervised, three-stage knowledge reinforcement training paradigm (“memorize–understand–reflect”) inspired by the Feynman learning technique, requiring neither continual pretraining nor human annotation. We introduce Self-Teaching, a novel strategy that automatically synthesizes knowledge-intensive tasks from raw documents. Furthermore, we construct Wiki-Newpages-2023-QA, the first benchmark explicitly designed to evaluate knowledge acquisition capability. Extensive experiments on models such as Llama2-7B demonstrate substantial improvements in memorization, extraction, and reasoning over newly encountered knowledge, while completely avoiding catastrophic forgetting. These results validate the efficacy of lightweight fine-tuning for dynamic knowledge integration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs current, existing approaches typically involve continued pre-training on new documents. However, they frequently face difficulties in extracting stored knowledge. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning, a learning framework aimed at improving an LLM's ability to effectively acquire new knowledge from unseen raw documents through self-teaching. Specifically, we develop a Self-Teaching strategy that augments the documents with a set of knowledge-intensive tasks created in a self-supervised manner, focusing on three crucial aspects: memorization, comprehension, and self-reflection. Additionally, we introduce three Wiki-Newpages-2023-QA datasets to facilitate an in-depth analysis of an LLM's knowledge acquisition ability concerning memorization, extraction, and reasoning. Extensive experimental results on various models, e.g., Llama2-7B reveal that Self-Tuning consistently exhibits superior performance across all knowledge acquisition tasks and excels in preserving previous knowledge.

Problem

• Enhance LLMs' knowledge acquisition from new documents
• Improve memorization, comprehension, and self-reflection in LLMs
• Address outdated information in one-time trained LLMs

Innovation

• Self-Tuning framework enhances LLMs
• Self-Teaching strategy with knowledge tasks
• Wiki-Newpages-2023-QA datasets for analysis