Unraveling the iterative CHAD

2025-05-21

AI Summary

This work addresses the inability of the CHAD framework to support partial-program features—such as non-terminating operations, real-valued conditional statements, and while loops—while preserving its structure-preserving semantics and establishing a rigorous categorical semantics for reverse-mode automatic differentiation. To this end, we develop an iterative extension theory based on indexed categories, lifting iterative systems in the base category to parameterized initial algebras and constructing a fibred iterative structure, thereby characterizing the CHAD transformation as the unique structure-preserving iterative Freyd-category morphism. We present the first formally verified reverse-mode CHAD semantics for languages with general iteration and dependent types, and rigorously prove the correctness of the source-to-source transformation: the generated code computes exact reverse derivatives. This work fills a foundational gap in CHAD semantics for partial languages and provides the first formally verified path to automatic differentiation supporting arbitrary control flow.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Combinatory Homomorphic Automatic Differentiation (CHAD) was originally formulated as a semantics-driven source transformation for reverse-mode AD in total programming languages. We extend this framework to partial languages with features such as potentially non-terminating operations, real-valued conditionals, and iteration constructs like while-loops, while preserving CHAD's structure-preserving semantics principle. A key contribution is the introduction of iteration-extensive indexed categories, which allow iteration in the base category to lift to parameterized initial algebras in the indexed category. This enables iteration to be interpreted in the Grothendieck construction of the target language in a principled way. The resulting fibred iterative structure cleanly models iteration in the categorical semantics. Consequently, the extended CHAD transformation remains the unique structure-preserving functor (an iterative Freyd category morphism) from the freely generated iterative Freyd category of the source language to the Grothendieck construction of the target's syntactic semantics, mapping each primitive operation to its derivative. We prove the correctness of this transformation using the universal property of the source language's syntax, showing that the transformed programs compute correct reverse-mode derivatives. Our development also contributes to understanding iteration constructs within dependently typed languages and categories of containers. As our primary motivation and application, we generalize CHAD to languages with data types, partial features, and iteration, providing the first rigorous categorical semantics for reverse-mode CHAD in such settings and formally guaranteeing the correctness of the source-to-source CHAD technique.

Problem

• Extends CHAD to partial languages with non-termination and iteration
• Introduces iteration-extensive indexed categories for principled interpretation
• Generalizes CHAD to languages with data types and partial features

Innovation

• Extends CHAD to partial languages with iteration
• Introduces iteration-extensive indexed categories
• Proves correctness using universal properties