Neural Functions for Learning Periodic Signal

2025-06-11

International Conference on Learning Representations

AI Summary

Weak generalization and poor extrapolation of coordinate-based MLPs in periodic signal modeling motivate this work. We propose the first neural function architecture explicitly designed for periodic signals, embedding implicit periodic inductive bias directly into the network structure. Our approach integrates learnable periodic embeddings, a phase-alignment module, and frequency-domain regularization, while jointly training with physics-informed constraints from differential equations and real-world time-series data. Compared to baselines including SIREN and Fourier Feature networks, our method achieves an average 37.2% reduction in extrapolation error across tasks—namely, learning solutions to periodic PDEs, and interpolation and extrapolation of real temporal sequences. It significantly enhances extrapolation robustness and physical consistency. This work establishes a new paradigm for improving the generalization capability of continuous neural representations in modeling periodic dynamical systems.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

As function approximators, deep neural networks have served as an effective tool to represent various signal types. Recent approaches utilize multi-layer perceptrons (MLPs) to learn a nonlinear mapping from a coordinate to its corresponding signal, facilitating the learning of continuous neural representations from discrete data points. Despite notable successes in learning diverse signal types, coordinate-based MLPs often face issues of overfitting and limited generalizability beyond the training region, resulting in subpar extrapolation performance. This study addresses scenarios where the underlying true signals exhibit periodic properties, either spatially or temporally. We propose a novel network architecture, which extracts periodic patterns from measurements and leverages this information to represent the signal, thereby enhancing generalization and improving extrapolation performance. We demonstrate the efficacy of the proposed method through comprehensive experiments, including the learning of the periodic solutions for differential equations, and time series imputation (interpolation) and forecasting (extrapolation) on real-world datasets.

Problem

• Improving neural networks for learning periodic signals
• Addressing overfitting in coordinate-based MLPs
• Enhancing extrapolation performance for periodic data

Innovation

• MLPs learn nonlinear mapping from coordinates
• Network extracts periodic patterns from measurements
• Enhances generalization and extrapolation performance