Identifiability and Order-Dimension Limits of In-Context Learning on Partial Orders

2026-08-14

AI Summary

This study addresses the identifiability and structural capacity boundaries of in-context learning with partial orders under limited prompts by establishing a dedicated theoretical framework. By integrating version space semantics with coordinate order equivalence theory, this work disentangles identifiability, teaching cost, and structural complexity. It proposes a trichotomy for precise prompt completion in open-world settings and establishes exact representation bounds for coordinate decoders. The research validates this query completion trichotomy, proving an upper bound on teaching size of n(n−1) and identifying dimension s as a necessary and sufficient condition for precise representation. Collectively, these findings provide a rigorous theoretical foundation for understanding the structural capacity of models performing in-context learning with partial orders under constrained prompting conditions.

Abstract

In-context learning is commonly formalized as inference from examples of a function. Partial orders instead combine transitivity, antisymmetry, and incomparability, so a finite prompt may not determine a queried comparison. We develop a theory of in-context learning on partial orders that separates logical identifiability, prompt teaching cost, structural complexity, and the exact capacity of a formal coordinate-decoder class. A version-space semantics makes background knowledge and open- versus closed-world assumptions explicit. For finite open-world prompts with positive and negative comparisons, we prove an exact completion trichotomy: after taking the reflexive transitive closure of the positive demonstrations, a query is forced true, forced false because every true completion creates a cycle or violates a negative demonstration, or remains genuinely ambiguous. For a known $n$-element universe, we characterize the open-world teaching number as the number of covers plus a blocker-set hitting number, prove that its maximum over all $n$-element posets is $n(n-1)$ and is uniquely attained by the antichain, and identify the blocker term as the exact cost of open-world rather than complete-Hasse semantics. We formalize prompt-dependent $s$-coordinate decoders and use the classical coordinate-order equivalence to obtain an exact representation boundary: dimension at most $s$ is necessary and sufficient, while width at most $s$ is a convenient sufficient condition.

Problem

• In-context learning
• Partial orders
• Identifiability
• Teaching cost
• Order dimension

Innovation

• In-context learning on partial orders
• Version-space semantics
• Open-world teaching number
• Coordinate-decoder capacity
• Order dimension