Learning Orthonormal Bases for Function Spaces

2026-05-19

AI Summary

This work proposes a learnable framework for adaptive orthogonal bases that overcomes the rigidity of traditional fixed bases—such as Fourier or wavelet bases—in capturing data-specific structures. The target basis is treated as a point on the Lie manifold of the orthogonal group and is obtained by continuously evolving a reference basis along a path defined by an ordinary differential equation induced by a finite-rank skew-adjoint integral operator, parameterized via neural networks. Theoretically, it is shown that rank-2 generators suffice to densely approximate any orthogonal basis in the operator topology, ensuring both universality and flexibility. Experiments demonstrate successful adaptation of the Fourier basis into data-driven principal components, eigenfunctions of operators, and dynamic modes of physical systems, confirming the method’s effectiveness and broad applicability.

Abstract

Infinite-dimensional orthonormal basis expansions play a central role in representing and computing with function spaces due to their favorable linear algebraic properties. However, common bases such as Fourier or wavelets are fixed and do not adapt to the structure of a given problem or dataset. In this paper, we aim to represent these bases with neural networks and optimize them. Our key idea is that any target infinite-dimensional orthonormal basis can be viewed either as a point on the Lie manifold of the orthogonal group, or equivalently, as the endpoint of a continuous path on that manifold that connects a reference basis, e.g. Fourier, to that target. Paths on the Lie manifold satisfy ordinary differential equations (ODEs) governed by skew-adjoint integral operators. Using neural networks to define finite-rank generators of such ODEs allows us to parameterize and optimize orthonormal bases in function space. While relying on finite-rank generators to model infinite operators might seem restrictive, we prove a universality result: even with a rank-2 generator, the integrated solutions of the ODE are dense in the orthogonal group under the appropriate operator topology. In other words, for any target orthonormal basis, there exists a path originating from a reference basis and driven by finite-rank generators that gets arbitrarily close to that target basis. We demonstrate the flexibility of our framework by transforming the Fourier basis into the principal components of a functional dataset, eigenfunctions of linear operators, or dynamic modes of energy-preserving physical simulations.

Problem

• orthonormal bases
• function spaces
• adaptive representation
• infinite-dimensional
• neural representation

Innovation

• orthonormal basis learning
• neural ODEs
• Lie manifold
• function space representation
• skew-adjoint operators