🤖 AI Summary
This work addresses the challenges in one-dimensional function approximation—namely, flat loss landscapes, slow convergence, and difficulty fitting high-frequency components—stemming from parameter symmetry in neural networks. To mitigate these issues, the authors propose a hardware-friendly input space expansion strategy that maps the original one-dimensional input into a higher-dimensional space by incorporating constants such as π. This approach effectively breaks parameter symmetry without increasing the number of model parameters. When combined with the L-BFGS optimizer, the method substantially enhances both training efficiency and approximation accuracy across a range of benchmark one-dimensional functions. Experimental results demonstrate that, with an optimal five-dimensional expansion using π as the expansion constant, the mean squared error is reduced by 66.3% and the average number of L-BFGS iterations decreases by 12%.
📝 Abstract
One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., $x$) with constant values (e.g., $\pi$) to form a higher-dimensional vector (e.g., $[\pi, \pi, x, \pi, \pi]$), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12\% on average) and enhances approximation accuracy (reducing final MSE by 66.3\% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with $\pi$ consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.