Leveraging Large Language Models for Tacit Knowledge Discovery in Organizational Contexts

2025-07-04

AI Summary

Organizations face significant challenges in identifying, acquiring, and structuring tacit knowledge—manifested by incomplete initial information, difficulty locating domain experts, entanglement of formal hierarchies and informal networks, and unclear questioning strategies. Method: This paper proposes a large language model (LLM)-driven agent framework integrating agent-based modeling (ABM), self-critical feedback mechanisms, and the Susceptible-Infectious (SI) model of knowledge diffusion. It iteratively reconstructs fragmented expertise through simulated employee interactions, enabling tacit knowledge recovery without direct expert engagement. Contribution/Results: Across 864 simulation trials, the framework achieves a 94.9% complete knowledge recall rate. Self-critical evaluation scores correlate strongly with external literature-based assessments (Pearson’s r > 0.92), demonstrating the method’s validity, robustness, and scalability for organizational knowledge engineering.

Technology Category

• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Documenting tacit knowledge in organizations can be a challenging task due to incomplete initial information, difficulty in identifying knowledgeable individuals, the interplay of formal hierarchies and informal networks, and the need to ask the right questions. To address this, we propose an agent-based framework leveraging large language models (LLMs) to iteratively reconstruct dataset descriptions through interactions with employees. Modeling knowledge dissemination as a Susceptible-Infectious (SI) process with waning infectivity, we conduct 864 simulations across various synthetic company structures and different dissemination parameters. Our results show that the agent achieves 94.9% full-knowledge recall, with self-critical feedback scores strongly correlating with external literature critic scores. We analyze how each simulation parameter affects the knowledge retrieval process for the agent. In particular, we find that our approach is able to recover information without needing to access directly the only domain specialist. These findings highlight the agent's ability to navigate organizational complexity and capture fragmented knowledge that would otherwise remain inaccessible.

Problem

• Documenting tacit knowledge in complex organizational structures
• Identifying knowledgeable individuals without direct access
• Reconstructing fragmented knowledge through iterative interactions

Innovation

• Agent-based framework using LLMs for knowledge discovery
• SI process modeling for knowledge dissemination analysis
• Self-critical feedback correlates with external evaluations