Abstract Compilation as Abstraction of Operator Semantics, applied to Cost Analysis

2026-08-10

AI Summary

Traditional least fixed-point semantics often fails to support precise static cost analysis due to its neglect of recursive structural information. This work proposes operator semantics as an intermediate representation bridging syntax and denotational semantics, treating programs as operators and constructing a higher-order abstract domain grounded in category theory, with composition as the core primitive. This framework enables abstract compilation that simultaneously achieves soundness, precision, and modularity. The approach supports cost analysis for general functional unknowns and generalized fold-based metrics, leveraging a solver-agnostic technique for extracting optimal recurrence relations. Consequently, it facilitates precise static cost analysis of recursive programs over algebraic data types, encompassing generalized size metrics beyond the reach of conventional methods.

Abstract

Least fixpoints are fundamental to program semantics, but they abstract away the recursive structure that generated them. We introduce operator semantics: a semantic intermediate representation between syntax and classical denotational semantics, which treats programs as operators. Abstract compilation is then understood as the act of abstracting such operators. We develop higher-order abstract domains for functions, operators, and programs themselves, in which composition is the key novel primitive, together with a categorical framework for constructing sound, precise, and modular abstract compilers. We instantiate this framework in the context of recurrence-based static cost analysis, developing solver-independent, optimal recurrence extraction techniques for recursive programs over algebraic data types, that support general function unknowns and catamorphic metrics, a broad class of size metrics beyond traditional approaches.

Problem

• cost analysis
• recursive programs
• abstract interpretation
• algebraic data types
• recurrence extraction

Innovation

• operator semantics
• abstract compilation
• higher-order abstract domains
• recurrence extraction
• catamorphic metrics