DispFormer: Pretrained Transformer for Flexible Dispersion Curve Inversion from Global Synthesis to Regional Applications

2025-01-08

AI Summary

Surface-wave dispersion curve inversion faces fundamental trade-offs between accuracy and efficiency, while deep learning approaches suffer from poor generalizability and heavy reliance on large-scale labeled datasets. Method: We propose the first pre-trained foundation model specifically designed for geophysical dispersion inversion. Our approach introduces a novel period-independent processing mechanism to support variable-length inputs and zero-/few-shot transfer; it leverages a Transformer architecture pre-trained on globally synthesized data, followed by zero-shot and few-shot fine-tuning strategies tailored to low signal-to-noise ratio, sparse measurements, and multi-regional settings. Contribution/Results: In zero-shot inference, predictions closely approximate ground truth; with only a handful of real-world labeled curves, performance surpasses conventional methods; empirical residuals are reduced, robustness is enhanced across diverse field conditions, and the model enables plug-and-play deployment without task-specific retraining.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Other Foundations of Computer Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Surface wave dispersion curve inversion is essential for estimating subsurface Shear-wave velocity ($v_s$), yet traditional methods often struggle to balance computational efficiency with inversion accuracy. While deep learning approaches show promise, previous studies typically require large amounts of labeled data and struggle with real-world datasets that have varying period ranges, missing data, and low signal-to-noise ratios. This study proposes DispFormer, a transformer-based neural network for inverting the $v_s$ profile from Rayleigh-wave phase and group dispersion curves. DispFormer processes dispersion data at each period independently, thereby allowing it to handle data of varying lengths without requiring network modifications or alignment between training and testing data. The performance is demonstrated by pre-training it on a global synthetic dataset and testing it on two regional synthetic datasets using zero-shot and few-shot strategies. Results indicate that zero-shot DispFormer, even without any labeled data, produces inversion profiles that match well with the ground truth, providing a deployable initial model generator to assist traditional methods. When labeled data is available, few-shot DispFormer outperforms traditional methods with only a small number of labels. Furthermore, real-world tests indicate that DispFormer effectively handles varying length data, and yields lower data residuals than reference models. These findings demonstrate that DispFormer provides a robust foundation model for dispersion curve inversion and is a promising approach for broader applications.

Problem

• Seismic Data Analysis
• Dispersion Curve Inversion
• Accuracy Improvement

Innovation

• DispFormer
• Pre-trained Model
• Seismic Data Analysis