Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge

2026-04-24

AI Summary

This work addresses the limitations of large language models in non-generative, task-specific applications, where their sequential generation mechanism hinders efficient representation of task-relevant knowledge. The authors propose a fully local adaptation method that requires no labeled data, human supervision, or model distillation, introducing the first task-agnostic Self-Knowledge Re-expression (SKR) mechanism. SKR leverages only the model’s intrinsic knowledge and unlabeled data to achieve efficient adaptation. Evaluated on financial document tasks, the approach demonstrates substantial performance gains: Recall@1 in information retrieval improves by over 40%, inference latency in object detection decreases by more than 76%, and AUPRC in anomaly detection increases by over 33%. Furthermore, it surpasses the current state-of-the-art retrieval models by at least 12.6% on the MMDocRAG benchmark.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialized, non-generative tasks. We attribute this performance bottleneck to the LLMs' knowledge expression mechanism, rather than to deficiencies in knowledge acquisition. To address this, we propose Self-Knowledge Re-expression (SKR), a novel, task-agnostic adaptation method. SKR transforms the LLM's output from generic token generation to highly efficient, task-specific expression. SKR is a fully local method that uses only unannotated data, requiring neither human supervision nor model distillation. Experiments on a large financial document dataset demonstrate substantial improvements: over 40% in Recall@1 for information retrieval tasks, over 76% reduction in object detection latency, and over 33% increase in anomaly detection AUPRC. Our results on the MMDocRAG dataset surpass those of leading retrieval models by at least 12.6%.

Problem

• large language models
• next-token prediction
• knowledge expression
• non-generative tasks
• task adaptation

Innovation

• Self-Knowledge Re-expression
• task-agnostic adaptation
• fully local method
• intrinsic knowledge
• next-token prediction