Tabular foundation model for GEOAI benchmark problems BM/AirportSoilProperties/2/2025

2025-09-03

AI Summary

This work addresses two geotechnical engineering tasks from the GEOAI benchmark BM/AirportSoilProperties/2/2025: (1) spatial prediction of undrained shear strength (su) from borehole profiles, and (2) imputation of missing mechanical parameters in dense site datasets. We introduce TabPFN—the first zero-shot/few-shot tabular foundation model—into geotechnical modeling, leveraging in-context learning and augmentation via large indirect databases, without fine-tuning or hyperparameter optimization. Compared to conventional hierarchical Bayesian models, our approach achieves higher accuracy and an order-of-magnitude faster inference for su spatial prediction; for parameter imputation, it yields lower RMSE and superior uncertainty calibration. This work establishes a novel, large-model–based paradigm for probabilistic site characterization, offering a scalable, low-barrier, general-purpose solution for intelligent geotechnical modeling.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper presents a novel application of the Tabular Prior-Data Fitted Network (TabPFN) - a transformer-based foundation model for tabular data - to geotechnical site characterization problems defined in the GEOAI benchmark BM/AirportSoilProperties/2/2025. Two tasks are addressed: (1) predicting the spatial variation of undrained shear strength (su) across borehole depth profiles, and (2) imputing missing mechanical parameters in a dense-site dataset. We apply TabPFN in a zero-training, few-shot, in-context learning setting - without hyper-parameter tuning - and provide it with additional context from the big indirect database (BID). The study demonstrates that TabPFN, as a general-purpose foundation model, achieved superior accuracy and well-calibrated predictive distributions compared to a conventional hierarchical Bayesian model (HBM) baseline, while also offering significant gains in inference efficiency. In Benchmark Problem #1 (spatial su prediction), TabPFN outperformed the HBM in prediction accuracy and delivered an order-of-magnitude faster runtime. In Benchmark Problem #2 (missing mechanical parameter imputation), TabPFN likewise achieved lower RMSE for all target parameters with well-quantified uncertainties, though its cumulative computation cost was higher than HBM's due to its one-variable-at-a-time inference. These results mark the first successful use of a tabular foundation model in geotechnical modeling, suggesting a potential paradigm shift in probabilistic site characterization.

Problem

• Predicting spatial variation of undrained shear strength
• Imputing missing mechanical parameters in geotechnical data
• Applying foundation model to geotechnical site characterization

Innovation

• TabPFN foundation model for tabular data analysis
• Zero-training few-shot learning with indirect context
• Superior accuracy and efficiency over Bayesian models