LMs as Task-Specific Knowledge Bases: An Interpretability Analysis

2026-06-25

AI Summary

This study investigates whether language models, unlike traditional knowledge bases, provide task-invariant consistent responses to the same factual query. Through behavioral experiments, parameter attribution analysis, and examination of chain-of-thought reasoning mechanisms, the work reveals for the first time that factual knowledge in such models is encoded in a task-specific manner: distinct tasks activate different subsets of model parameters, thereby hindering the cross-task sharing of learned facts. These findings challenge the prevailing assumption that model parameters constitute a unified knowledge repository and instead demonstrate that knowledge representation is tightly coupled with the manner of querying. The study thus offers a novel perspective on the internal organization of knowledge in large language models.

Abstract

Language models (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important property of knowledge bases is that different queries for the same fact return consistent results, drawing on a single source of truth. We investigate whether LMs satisfy this property through behavioral and mechanistic analyses. Our results suggest that they encode knowledge in a task-specific manner. Behaviorally, facts acquired on one task frequently fail to co-emerge on others during training. Parameter localization experiments suggest a mechanistic explanation, revealing distinct parameter subsets underlying different tasks for the same fact. Finally, we show that chain-of-thought reasoning draws part of its effectiveness from engaging task-specific parameters beyond those tied to the evaluation task. Our findings suggest that what the model knows and how it is asked are intertwined in parameter space, undermining the "knowledge base" analogy and carrying implications for the reliability and controllability of factual knowledge in LMs.

Problem

• language models
• knowledge base
• task-specific knowledge
• factual consistency
• parameter space

Innovation

• task-specific knowledge
• parameter localization
• chain-of-thought reasoning
• knowledge consistency
• mechanistic interpretability