Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion

2024-12-27

AI Summary

Seismic full-waveform inversion (FWI) suffers from poor generalization across diverse geological scenarios and prohibitive computational costs. To address these challenges, this work pioneers the integration of task-agnostic deep learning foundation models into FWI, coupled with parameter-efficient fine-tuning (PEFT) techniques—such as LoRA and Adapter—for lightweight adaptation. The proposed method optimizes fewer than 1% of the model parameters, drastically reducing memory footprint and computational overhead while maintaining—or even surpassing—the accuracy of full-parameter fine-tuning. Crucially, it demonstrates superior performance on out-of-distribution geological settings compared to both conventional task-specific models and fully fine-tuned counterparts. This study breaks the long-standing “one-task, one-model” paradigm in FWI, establishing a novel pathway toward developing general-purpose, transferable geophysical inversion foundation models.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Seismic full waveform inversion (FWI) has seen promising advancements through deep learning. Existing approaches typically focus on task-specific models trained and evaluated in isolation that lead to limited generalization across different geological scenarios. In this work we introduce a task-agnostic foundational model for FWI that captures general features across tasks. We first demonstrate that full fine-tuning of this foundational model outperforms task-specific models built from scratch by delivering superior performance across multiple benchmarks. Building upon this we employ parameter-efficient fine-tuning (PEFT) to further reduce computational overhead. By fine-tuning only a small fraction of the model parameters PEFT achieves comparable results to full fine-tuning while significantly lowering memory and computational requirements. Additionally, PEFT excels in out-of-distribution tasks where it outperforms both full fine-tuning and task-specific models. These findings establish the value of foundational modeling for FWI and highlight PEFT as an effective strategy for efficient and scalable adaptation across diverse tasks.

Problem

• Seismic Full Waveform Inversion (FWI)
• Deep Learning Model Optimization
• Geological Variability

Innovation

• Deep Learning Model
• Parameter Efficient Fine-Tuning (PEFT)
• Seismic Full Waveform Inversion (FWI)