HPC-Enabled Video-based Coastal Wave Parameter Estimation Using V-JEPA and Deep Spatiotemporal Learning

2026-07-13

AI Summary

This study addresses the challenges of high cost, limited spatial coverage, and susceptibility to adverse weather that plague conventional in situ coastal wave observation methods, which hinder efficient acquisition of wave parameters. To overcome these limitations, the authors propose a deep learning framework that leverages monocular coastal video to jointly estimate five key wave parameters—significant wave height, maximum wave height, peak period, zero-crossing period, and wave direction—under data-scarce conditions. The approach integrates a self-supervised V-JEPA vision transformer, a SlowFast dual-stream temporal encoder, and Farneback optical flow, while incorporating Airy dispersion relation constraints to enforce physical consistency. Validated with only six annotated scenes and accelerated via high-performance computing, the model achieves Pearson correlation coefficients ranging from 0.451 to 0.832 across all five parameters, demonstrating both feasibility and strong cross-site generalization capability.

Abstract

High deployment cost, poor spatial coverage and susceptibility to storm conditions are all challenges faced by traditional in-situ methods. This paper presents a video-based and high performance computing (HPC) enabled deep learning framework for joint sensor free estimation of five coastal wave parameters, namely significant wave height (Hs), maximum wave height (Hmax), peak period (Tp), zero upcrossing period (Tz) and wave direction (theta) from monocular coastal video. The proposed architecture comprises of a V-JEPA (self supervised) ViT Small backbone for robust spatiotemporal feature extraction in visually challenging scenarios, a dual-stream SlowFast temporal encoder for broad bandwidth representation of wave motion in both hydrodynamic breaking and swell regimes, an optical flow stream based on Farneback optical flow algorithm for adding saliency information to the structure with emphasis on hydrodynamically active wavelength bands of waves, and a multi-task regression layer with dispersion constraints (Airy wave dispersion lambda_p = 0.1). The model was trained on an NVIDIA DGX A100 cluster and was early stopped at epoch 31 and achieved Pearson correlation coefficients of 0.451, 0.578, 0.643, 0.680 and 0.832 for Hs, Hmax, Tp, Tz and wave direction respectively, with generalization ability to geographically diverse held out test data sites. While operating in a data-limited regime (6 annotated training scenes), the framework demonstrates statistically significant temporal correlations (PCC of 0.451 to 0.832), confirming proof of concept feasibility; R2 values (max 0.246) indicate that variance capture will improve with larger annotated datasets.

Problem

• coastal wave parameter estimation
• sensor-free measurement
• video-based monitoring
• spatiotemporal learning
• wave dynamics

Innovation

• V-JEPA
• SlowFast
• optical flow
• multi-task regression with dispersion constraints
• HPC-enabled deep learning