Simply Typed Reverse-Mode AD with Variants: Denotational Correctness via Idempotent Completion

2026-07-15

AI Summary

This work addresses the challenge of modeling variant types in reverse-mode automatic differentiation, where runtime-dependent cotangent spaces resist straightforward representation in simple type systems. The authors propose embedding the cotangent fibers of each source type into a unified environment type and employ idempotent projectors indexed by primal values to select active fibers, thereby achieving—within a non-dependent type language—an equivalent characterization of dependent cotangent families. Leveraging categorical constructions such as Karoubi completion, coproducts, and idempotent splitting, they develop a bicartesian closed categorical semantics for variant types. This framework requires only ordinary types, projectors, and backpropagators to recover existing dependent semantics and establishes a categorical equivalence between idempotent completion and constant-family models.

Abstract

Reverse-mode automatic differentiation is commonly given a denotational account in which each source type has a single cotangent type. Variant types obstruct this simply typed representation because the valid cotangent space depends on the branch selected at run time. Existing correctness results therefore use primal-indexed families of cotangent spaces, whose natural internal language is dependently typed. We show that the same dependency can be represented in an ordinary nondependent target. The cotangent fibres of each source type are embedded in a common ambient type, and a primal-indexed idempotent selects the valid fibre. Semantically, this amounts to passing from the constant-family model to its Karoubi completion. For a category $\mathcal C$ and a regular infinite cardinal $κ$, we prove that the constant-family inclusion extends to an equivalence $\mathrm{Kar}(\mathrm{Copow}*κ(\mathcal C)) \simeq \mathrm{Fam}*κ(\mathcal C)$ precisely when $\mathcal C$ is Cauchy complete and every $κ$-small family admits a common retract host. We also construct the resulting coproducts explicitly. Applying this theorem, we obtain a bicartesian closed semantics for reverse-mode automatic differentiation with variants using only ordinary target types, projectors, and backpropagators. Splitting the generated idempotents recovers the established dependent semantics. Thus dependent cotangent families and simply typed ambient cotangents equipped with projectors are equivalent presentations of the same denotational transformation.

Problem

• reverse-mode automatic differentiation
• variant types
• denotational semantics
• simply typed lambda calculus
• cotangent spaces

Innovation

• reverse-mode automatic differentiation
• variants
• idempotent completion
• Karoubi completion
• denotational semantics