🤖 AI Summary
This paper addresses the issues of model redundancy and poor interpretability in Linear Recurrent Neural Networks (LRNNs) for time-series approximation. We propose Predictive Linear Recurrent Neural Networks (PLRNNs), which jointly optimize network weights and sparse architecture via least-squares solving of linear systems—enabling, for the first time, integrated parameter learning and structural pruning. PLRNNs exhibit elliptical dynamical trajectories, ensuring both interpretability and compact functional representation. Structural pruning is guided by principal component analysis, significantly reducing model size. On the Multi-Sinusoidal Oscillator (MSO) benchmark, PLRNN achieves state-of-the-art performance with the fewest neurons. Furthermore, it demonstrates strong generalization on real-world tasks: robotic soccer motion modeling and stock price prediction.
📝 Abstract
Recurrent neural networks are a powerful means to cope with time series. We show how a type of linearly activated recurrent neural networks, which we call predictive neural networks, can approximate any time-dependent function f(t) given by a number of function values. The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, the network size can be reduced by taking only most relevant components. Thus, in contrast to others, our approach not only learns network weights but also the network architecture. The networks have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several experiments, among them multiple superimposed oscillators (MSO), robotic soccer, and predicting stock prices. Predictive neural networks outperform the previous state-of-the-art for the MSO task with a minimal number of units.
