TabPFN Extensions for Interpretable Geotechnical Modelling

2026-03-21

AI Summary

This study addresses the challenge of parameter inference from sparse and heterogeneous borehole data in geotechnical engineering by introducing TabPFN—a Transformer-based tabular foundation model—into geotechnical modeling for the first time. Leveraging in-context learning, the approach enables Bayesian iterative imputation of missing mechanical parameters and soil type classification without retraining, supporting interpretable inference. Through cosine similarity embedding visualization, posterior distribution estimation, and SHAP analysis, the method reveals unsupervised separation of soil classes and uncovers physically meaningful inter-parameter relationships. It successfully distinguishes clay from sand and significantly improves imputation accuracy for four key mechanical parameters, with posterior uncertainties aligning with established geotechnical principles and reproducing classical empirical relationships such as the Skempton compression index.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Geotechnical site characterisation relies on sparse, heterogeneous borehole data where uncertainty quantification and model interpretability are as critical as predictive accuracy for reliable engineering decisions. This paper presents an exploratory investigation into the use of TabPFN, a transformer-based tabular foundation model using in-context learning, and its extension library tabpfn-extensions for two geotechnical inference tasks: (1) soil-type classification using N-value and shear-wave velocity data from a synthetic geotechnical dataset, and (2) iterative imputation of five missing mechanical parameters ($s_\mathrm{u}$, $E_{\mathrm{u}}$, ${σ'}_\mathrm{p}$, $C_\mathrm{c}$, $C_\mathrm{v}$) in benchmark problem BM/AirportSoilProperties/2/2025. We apply cosine-similarity analysis to TabPFN-derived embeddings, visualise full posterior distributions from an iterative inference procedure, and compute SHAP-based feature importance, all without model retraining. Learned embeddings clearly separate Clay and Sand samples without explicit soil-type supervision; iterative imputation improves predictions for four of five target parameters, with posterior widths that reflect physically reasonable parameter-specific uncertainty; and SHAP analysis reveals the inter-parameter dependency structure, recovering established geotechnical relationships including the Skempton compression index correlation and the inverse dependence of preconsolidation pressure on water content. These results suggest the potential of foundation-model-based tools to support interpretable, uncertainty-aware parameter inference in data-scarce geotechnical practice.

Problem

• geotechnical modelling
• uncertainty quantification
• model interpretability
• sparse borehole data
• parameter inference

Innovation

• TabPFN
• in-context learning
• interpretable modeling
• uncertainty quantification
• geotechnical parameter imputation