Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence

2025-04-21

AI Summary

This work addresses the poor generalizability of data-driven subgrid-scale (SGS) models for oceanic turbulence when transferred across dynamical regimes. We propose an interpretable transfer learning framework grounded in Fourier spectral analysis. Methodologically, we design a nine-layer convolutional neural network (CNN) SGS model and integrate spectral decomposition with single-layer fine-tuning to enable cross-regime learning between two quasi-geostrophic turbulent systems. Key contributions include: (i) empirical identification that CNN kernels universally learn low-pass, Gabor, and high-pass filters; and (ii) demonstration that transfer learning achieves physically consistent spectral matching by correcting systematic underestimation bias in the predicted power spectrum. Remarkably, fine-tuning only a single layer suffices to precisely reproduce the target regime’s spectral characteristics. This approach significantly enhances generalization accuracy for unseen turbulent states—even under limited training data—thereby providing both theoretical insight and a practical paradigm for interpretable, physics-aware parameterization transfer in dynamical systems.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems.

Problem

• Enhancing neural network performance in turbulence modeling using transfer learning
• Analyzing NN generalization to unseen ocean turbulence regimes
• Correcting spectral underestimation in NNs via targeted layer retraining

Innovation

• Uses 9-layer CNN for subgrid forcing prediction
• Employs Fourier analysis to examine NN kernels
• Retrains one layer to correct spectral underestimation