Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing

2026-02-25

AI Summary

This work addresses the limited robustness of continuous-time sequence models under missing inputs by introducing a learnable, structured oscillatory dynamics mechanism into Closed-form Continuous (CfC) networks. The proposed approach employs a spiking module that generates state-dependent sinusoidal oscillations and integrates recurrent self-attention to enable biologically inspired internal temporal modeling. Rather than relying on stochastic perturbations, robustness to input interruptions is enhanced through an explicit oscillatory architecture. Evaluated on the sMNIST task with multiple gaps, the spiking variant achieves a 4.62 percentage point accuracy gain over the baseline (Cohen’s d = 0.87), while the self-attention variant improves performance by 2.78 percentage points (p = 0.041), both significantly outperforming a random-perturbation control condition.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Probabilistic Circuits and Graphical Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We introduce PDNA (Pulse-Driven Neural Architecture), a method for augmenting continuous-time recurrent networks with learnable oscillatory dynamics that maintain internal state evolution independently of external input. Built on Closed-form Continuous-time (CfC) networks, PDNA adds two components: (1) a pulse module that generates structured oscillations $A \cdot \sin(ωt + \varphi(h))$ with learnable frequencies and state-dependent phase, and (2) a self-attend module that applies recurrent self-attention to the hidden state. Through a controlled ablation study on sequential MNIST (sMNIST) with five random seeds, we evaluate gap robustness -- the ability to maintain performance when portions of the input sequence are removed at test time. Our key finding is that structured oscillatory dynamics significantly improve robustness to input interruptions: the self-attend variant achieves a statistically significant 2.78 percentage point multi-gap advantage over baseline ($p = 0.041$), while the pulse variant shows a 4.62 pp advantage with large effect size (Cohen's $d = 0.87$). A noise control (random perturbation of equal magnitude) provides no benefit, confirming that the advantage is structural rather than merely dynamic. These results provide evidence that continuous-time models can benefit from biologically-inspired internal oscillatory mechanisms for temporal robustness.

Problem

• robustness
• continuous-time sequence processing
• input gaps
• oscillatory dynamics
• temporal robustness

Innovation

• oscillatory dynamics
• continuous-time recurrent networks
• pulse-driven architecture
• temporal robustness
• self-attention