An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

2026-06-23

AI Summary

This study addresses the ill-posed inverse problem of simultaneously retrieving soil moisture (SM), leaf area index (LAI), and plant height (PH) at the field scale, which is hindered by ambiguous multimodal remote sensing responses arising from soil–canopy coupling and the limited performance of conventional models in heterogeneous smallholder agricultural landscapes. To overcome these challenges, the authors propose an iterative energy-based transformer (iEBT) that fuses time-series Sentinel-1 SAR and Sentinel-2 optical data. The method employs shared sequential modeling and a learnable scalar compatibility energy function, optimizing parameters via normalized gradient descent iterations. It introduces an energy-driven joint inversion framework and leverages the terminal energy value as a calibration-free posterior quality indicator. Evaluated on 700 ground samples from Varanasi, India, the approach achieves a mean R² of 0.854 across four crops (SM: 0.841, LAI: 0.905, PH: 0.821), with RMSE significantly reduced after excluding the top 10% high-energy samples, demonstrating its reliability and robustness.

Abstract

Field-scale retrieval of surface soil moisture (SM), leaf area index (LAI), and plant height (PH) is essential for precision agriculture, yet it remains an ill-posed inverse problem. Concurrent variations in soil moisture and canopy density generate substantial ambiguities in radar backscatter and spectral responses, which reduces the effectiveness of traditional feedforward regression models in heterogeneous smallholder cropping systems. This study presents the Iterative Energy-Based Transformer (iEBT) for the joint retrieval of coupled soil-canopy states from Sentinel-1 C-band SAR and Sentinel-2 multispectral time series. Instead of direct regression, iEBT embeds multi-modal predictors within a shared sequence, produces an initial state estimate, and iteratively updates the target [SM, LAI, PH] vector through normalized gradient descent to minimize a learned scalar compatibility energy function. Using 700 quality-controlled field measurements from Varanasi, India, iEBT achieved the highest learned-model performance on the random test split, with a four-seed mean R^2 of 0.854 \pm 0.012 (R_SM^2 = 0.841, R_LAI^2 = 0.905, R_PH^2 = 0.821). WCM and PROSAIL were retained as physically interpretable SAR and optical reference models for comparison. Modality ablations confirmed that Sentinel-1 drives SM retrieval, while Sentinel-2 dominates LAI, whereas PH relies on combined structural-phenological signatures. Crucially, the model's terminal energy functions as an uncalibrated post-retrieval quality diagnostic; screening the 10% highest-energy samples markedly reduced target level root-mean-square errors. While leave-one-campaign-out validation highlights persistent cross-season domain shift challenges due to localized management variations, compatibility-guided multimodal fusion offers a structured self-diagnostic path toward reliable biophysical parameter estimation

Problem

• soil moisture
• leaf area index
• plant height
• multimodal retrieval
• ill-posed inverse problem

Innovation

• energy-based model
• multimodal transformer
• iterative retrieval
• compatibility energy
• joint biophysical parameter estimation