Derivation Depth as an Information Metric: Axioms, Coding Theorems, and Storage--Computation Tradeoffs

2026-02-22

AI Summary

This work investigates how to quantify the reasoning effort required to answer a query and achieve an optimal trade-off between storage and computation. To this end, it introduces “derivation depth” as a computable information-theoretic measure, leveraging a two-layer model that separates knowledge from its physical representation. By integrating reasoning trajectory encoding with the incompressibility principle from information theory, the paper establishes fundamental bounds linking query description complexity to derivation depth. The core contributions include the first formalization of derivation depth, the revelation of its intrinsic connection to minimum description length, a proof of its computability, and the derivation of a lower bound on description length that scales logarithmically with knowledge base size. Furthermore, the study proposes an optimal caching allocation strategy with approximation guarantees, extendable to settings involving noisy or incomplete knowledge bases.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce derivation depth-a computable metric of the reasoning effort needed to answer a query based on a given set of premises. We model information as a two-layered structure linking abstract knowledge with physical carriers, and separate essential core facts from operational shortcuts. For any finite premise base, we define and prove the computability of derivation depth. By encoding reasoning traces and applying information-theoretic incompressibility arguments, we establish fundamental bounds linking depth to the descriptive complexity of queries. For frequently asked, information-rich queries, the minimal description length grows proportionally to depth times the logarithm of the knowledge base size. This leads to a practical storage-computation tradeoff: queries accessed beyond a critical threshold become cheaper to cache than recompute. We formulate optimal cache allocation as a mathematical optimization problem solvable with approximation guarantees and extend the framework to handle noisy or incomplete knowledge bases.

Problem

• derivation depth
• information metric
• storage-computation tradeoff
• reasoning effort
• knowledge base

Innovation

• derivation depth
• information metric
• storage-computation tradeoff
• descriptive complexity
• reasoning trace