Toward Effective and Reliable LLM Agents via Dynamic Ontology

2026-08-24

AI Summary

本文通过动态构建和优化任务导向的本体来解决大语言模型在特定领域任务中的知识利用不全和多步决策脆弱的问题。

Abstract

Large language model (LLM) agents rely heavily on knowledge encoded in model parameters or presented as unstructured context. In domain-specific tasks, this leaves important semantic connections implicit. This often results in incomplete evidence use and brittle multi-step decisions. Ontologies offer a way to externalize domain concepts and relations as machine-interpretable structures, but constructing task-usable ontologies traditionally requires substantial effort from domain experts and is difficult to scale. Automatic construction is also challenging: an ontology that appears semantically plausible may not contain the relational structures needed for actual decision making. We present OaK, an ontology-as-a-kernel framework that dynamically constructs and refines task-oriented ontologies for LLM agents. Given task requirements and training data, OaK constructs an ontology and its knowledge graph, generates task-adaptation functions for graph reasoning, and uses judge feedback to iteratively refine both. By making relevant concepts and relations explicit, the ontology grounds knowledge retrieval and multi-step decision making. We evaluate OaK on TravelPlanner, CRMArenaPro, and ToolQA. Results show that OaK improves standard LLM agents, strengthens evidence grounding, and boosts the reliability of multi-step reasoning.

Problem

• Large Language Model (LLM) Agents
• Semantic Connections
• Multi-step Decisions
• Ontologies
• Knowledge Retrieval

Innovation

• dynamic ontology construction
• task-oriented ontologies
• graph reasoning
• iterative refinement