FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences

2026-05-09

AI Summary

This work addresses the inherent trade-off in existing state space models between preserving long-term memory and capturing high-resolution recent dynamics, a limitation rooted in the HiPPO operator’s conflict between time-scale invariance and sensitivity to local temporal variations. To resolve this, the study introduces fractional measure theory into state space modeling for the first time, proposing a recursive memory update architecture based on a tunable singularity-index projection operator. This design retains scale-invariant memory structures while enhancing responsiveness to recent perturbations. Furthermore, by diagonalizing the state space, the model enables joint multi-scale temporal feature learning. Evaluated on the Long Range Arena benchmark, the method achieves an average accuracy of 87.11%, including 61.85% on ListOps, outperforming state-of-the-art models such as S5.

Abstract

Effective sequence modeling fundamentally requires balancing the retention of unbounded history with the high-resolution detection of abrupt short-term variations common in real-world phenomena. However, existing state space models (SSMs) relying on high-order polynomial projection operators (HiPPO) face a critical trade-off where uniform measures dilute recent information to maintain timescale invariance, while exponential measures sacrifice global context to capture local dynamics. This paper proposes a Fractional Recurrent Architecture for Computational Temporal Analysis of Long sequences (FRACTAL), a novel architecture integrating fractional measure theory into recursive memory updates to address this limitation. By deriving projection operators with analytically characterized spectral properties and a tunable singularity index, the proposed method amplifies sensitivity to recent signal perturbations while preserving the spectral structure that encodes scale-invariant memory dynamics. This theoretical innovation is instantiated within a simplified diagonalized state space framework by modulating input projection initialization to enable simultaneous capture of multi-scale temporal features. FRACTAL achieves an average score of 87.11\% on the Long Range Arena benchmark, including 61.85\% on the ListOps task, outperforming the S5 model.

Problem

• state space models
• long sequence modeling
• temporal dynamics
• timescale invariance
• short-term variations

Innovation

• Fractional Measure Theory
• State Space Models
• Multi-scale Temporal Modeling
• Projection Operators
• Long Sequence Modeling