A Soft Sensor Method with Uncertainty-Awareness and Self-Explanation Based on Large Language Models Enhanced by Domain Knowledge Retrieval

2025-01-06

AI Summary

Industrial soft sensor modeling faces critical bottlenecks—including high development costs, poor robustness, training instability, and limited interpretability. Method: This work pioneers the integration of large language models (LLMs) with in-context learning (ICL) into soft sensing, departing from conventional supervised learning. We propose an LLM-driven zero-shot auxiliary variable selection and uncertainty-aware few-shot prediction framework that generates natural-language explanations and probabilistic uncertainty quantification. Our approach innovatively combines industrial knowledge vector retrieval with structured-data-to-text encoding, and employs Monte Carlo prompt sampling for uncertainty calibration. Contribution/Results: Evaluated on multiple industrial datasets, the method achieves state-of-the-art (SOTA) accuracy, significantly enhances generalization and robustness, eliminates training instability entirely, and enables zero-shot deployment and trustworthy decision-making.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Data-driven soft sensors are crucial in predicting key performance indicators in industrial systems. However, current methods predominantly rely on the supervised learning paradigms of parameter updating, which inherently faces challenges such as high development costs, poor robustness, training instability, and lack of interpretability. Recently, large language models (LLMs) have demonstrated significant potential across various domains, notably through In-Context Learning (ICL), which enables high-performance task execution with minimal input-label demonstrations and no prior training. This paper aims to replace supervised learning with the emerging ICL paradigm for soft sensor modeling to address existing challenges and explore new avenues for advancement. To achieve this, we propose a novel framework called the Few-shot Uncertainty-aware and self-Explaining Soft Sensor (LLM-FUESS), which includes the Zero-shot Auxiliary Variable Selector (LLM-ZAVS) and the Uncertainty-aware Few-shot Soft Sensor (LLM-UFSS). The LLM-ZAVS retrieves from the Industrial Knowledge Vector Storage to enhance LLMs' domain-specific knowledge, enabling zero-shot auxiliary variable selection. In the LLM-UFSS, we utilize text-based context demonstrations of structured data to prompt LLMs to execute ICL for predicting and propose a context sample retrieval augmentation strategy to improve performance. Additionally, we explored LLMs' AIGC and probabilistic characteristics to propose self-explanation and uncertainty quantification methods for constructing a trustworthy soft sensor. Extensive experiments demonstrate that our method achieved state-of-the-art predictive performance, strong robustness, and flexibility, effectively mitigates training instability found in traditional methods. To the best of our knowledge, this is the first work to establish soft sensor utilizing LLMs.

Problem

• data-driven soft sensor modeling
• In-Context Learning (ICL)
• large language models

Innovation

• LLM-FUESS
• In-Context Learning
• Uncertainty-aware Few-shot Soft Sensor