🤖 AI Summary
This work addresses the fragmentation in current linear recurrent neural network (LRNN) research, which is hindered by scattered implementations, strong framework dependencies, and a lack of standardized interfaces—impeding reproducibility, comparison, and extension. To bridge this gap, we introduce lrnnx, the first unified open-source library that integrates multiple modern LRNN architectures. Built on mainstream deep learning frameworks, lrnnx supports diverse parameterizations and discretization schemes without requiring custom CUDA kernels. The library offers a multi-level modular interface, ranging from low-level components to high-level models, substantially lowering the barrier to both usage and development. This design enables flexible control, facilitates fair benchmarking, and establishes a much-needed standardized toolkit for the LRNN community.
📝 Abstract
Linear recurrent neural networks (LRNNs) provide a structured approach to sequence modeling that bridges classical linear dynamical systems and modern deep learning, offering both expressive power and theoretical guarantees on stability and trainability. In recent years, multiple LRNN-based architectures have been proposed, each introducing distinct parameterizations, discretization schemes, and implementation constraints. However, existing implementations are fragmented across different software frameworks, often rely on framework-specific optimizations, and in some cases require custom CUDA kernels or lack publicly available code altogether. As a result, using, comparing, or extending LRNNs requires substantial implementation effort. To address this, we introduce $\texttt{lrnnx}$, a unified software library that implements several modern LRNN architectures under a common interface. The library exposes multiple levels of control, allowing users to work directly with core components or higher-level model abstractions. $\texttt{lrnnx}$ aims to improve accessibility, reproducibility, and extensibility of LRNN research and applications. We make our code available under a permissive MIT license.