Fast and General Automatic Differentiation for Finite-State Methods

2026-02-11

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Lexical Semantics and Morphology

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We propose a new method, that we coined the ``morphism-trick'', to integrate custom implementations of vector-Jacobian products in automatic differentiation softwares, applicable to a wide range of semiring-based computations. Our approach leads to efficient and semiring-agnostic implementations of the backward pass of dynamic programming algorithms. For the particular case of finite-state methods, we introduce an algorithm that computes and differentiates the $\oplus$-sum of all paths'weight of a finite-state automaton. Results show that, with minimal effort from the user, our novel library allows computing the gradient of a function w.r.t. to the weights of a finite state automaton orders of magnitude faster than state-of-the-art automatic differentiation systems. Implementations are made available via an open-source library distributed under a permissive license.

Problem

• automatic differentiation
• finite-state automaton
• vector-Jacobian product
• semiring
• dynamic programming

Innovation

• morphism-trick
• automatic differentiation
• finite-state automata
• semiring-based computation
• vector-Jacobian products