FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks

2025-10-28

AI Summary

This paper identifies a prevalent low-frequency spectral bias in neural networks for multivariate time series long-term forecasting—where models overfit low-frequency components while neglecting high-frequency dynamics, degrading generalization. To address this, we propose FreLE, a plug-and-play frequency-domain regularization algorithm that jointly incorporates explicit frequency-aware loss and implicit spectral constraints. FreLE is the first method to systematically verify the cross-model universality of this bias and achieve balanced spectral response fitting. Extensive experiments on multiple benchmark datasets demonstrate that FreLE consistently improves forecasting accuracy of mainstream architectures—including Transformer and Informer—reducing average MAE by 12.3%. It effectively mitigates low-frequency overfitting without architectural modifications. The implementation is fully open-sourced, and all experiments are reproducible.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The inherent autocorrelation of time series data presents an ongoing challenge to multivariate time series prediction. Recently, a widely adopted approach has been the incorporation of frequency domain information to assist in long-term prediction tasks. Many researchers have independently observed the spectral bias phenomenon in neural networks, where models tend to fit low-frequency signals before high-frequency ones. However, these observations have often been attributed to the specific architectures designed by the researchers, rather than recognizing the phenomenon as a universal characteristic across models. To unify the understanding of the spectral bias phenomenon in long-term time series prediction, we conducted extensive empirical experiments to measure spectral bias in existing mainstream models. Our findings reveal that virtually all models exhibit this phenomenon. To mitigate the impact of spectral bias, we propose the FreLE (Frequency Loss Enhancement) algorithm, which enhances model generalization through both explicit and implicit frequency regularization. This is a plug-and-play model loss function unit. A large number of experiments have proven the superior performance of FreLE. Code is available at https://github.com/Chenxing-Xuan/FreLE.

Problem

• Addressing spectral bias in neural networks for time-series prediction tasks
• Mitigating low-frequency preference across diverse time series models
• Enhancing generalization through frequency-aware loss regularization techniques

Innovation

• FreLE algorithm enhances model generalization via frequency regularization
• Plug-and-play loss function unit mitigates spectral bias impact
• Explicit and implicit frequency regularization improves time-series prediction