Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction

2025-05-28

AI Summary

Traditional concrete creep prediction treats time as a static input, neglecting its inherent temporal evolution. This work reformulates the task as an autoregressive time-series forecasting problem—the first such formulation in this domain—and proposes a novel three-stream attention Transformer architecture incorporating temporal, feature, and batch attention mechanisms to overcome limitations of static modeling. The method integrates SHAP-based interpretability analysis with attention pooling, jointly optimizing predictive accuracy and engineering interpretability, and enables real-time web deployment. Evaluated on 160 days of field-measured data, it achieves a MAPE of 1.63% and R² = 0.999, substantially outperforming empirical models and state-of-the-art machine learning approaches. SHAP analysis identifies Young’s modulus as the most influential factor. This study represents the first successful adaptation and systematic application of the Transformer architecture to concrete creep prediction.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

This paper presents a novel Triple Attention Transformer Architecture for predicting time-dependent concrete creep, addressing fundamental limitations in current approaches that treat time as merely an input parameter rather than modeling the sequential nature of deformation development. By transforming concrete creep prediction into an autoregressive sequence modeling task similar to language processing, our architecture leverages the transformer's self-attention mechanisms to capture long-range dependencies in historical creep patterns. The model implements a triple-stream attention framework incorporating temporal attention for sequential progression, feature attention for material property interactions, and batch attention for inter-sample relationships. Evaluated on experimental datasets with standardized daily measurements spanning 160 days, the architecture achieves exceptional performance with mean absolute percentage error of 1.63% and R2 values of 0.999 across all datasets, substantially outperforming traditional empirical models and existing machine learning approaches. Ablation studies confirm the critical role of attention mechanisms, with attention pooling contributing most significantly to model performance. SHAP analysis reveals Young's modulus as the primary predictive feature, followed by density and compressive strength, providing interpretability essential for engineering applications. A deployed web-based interface facilitates practical implementation, enabling real-time predictions using standard laboratory parameters. This work establishes the viability of applying transformer architectures to materials science problems, demonstrating the potential for data-driven approaches to revolutionize structural behavior prediction and engineering design practices.

Problem

• Predict time-dependent concrete creep using transformers
• Model sequential deformation via autoregressive attention mechanisms
• Improve accuracy over traditional empirical creep prediction methods

Innovation

• Triple Attention Transformer for creep prediction
• Autoregressive sequence modeling like language processing
• Three attention streams capture diverse dependencies