LLMs on Tabular Data with Limited Semantics: Evidence from Industrial Car Retrofit Prediction

2026-06-13

AI Summary

This study addresses the challenge of predicting whether a new vehicle prototype requires modification, along with the modification type and duration, using semantic-sparse and privacy-constrained structured industrial data. The authors systematically evaluate classical tree-based models against various large language model (LLM) strategies—including feature embedding, direct prompting for classification, and ML+LLM stacking—on row-serialized tabular data. Results show that tree-based models remain the strongest standalone baseline; LLM embeddings achieve an AUC of 0.982 in binary classification; and hybrid stacking significantly improves multiclass performance (F1 = 0.626). Moreover, conventional models augmented with time-lagged features outperform basic temporal models such as Chronos-small. The findings suggest that in low-semantic industrial tabular settings, LLMs are better suited as enhancement components rather than replacements, and demonstrate the efficacy of fusion architectures.

Abstract

Industrial retrofit planning depends on structured operational data rather than free text: planners must estimate whether a newly registered prototype will require a retrofit, which retrofit package it will need, and how long the work will take. We study an industrial dataset linking a prototype-registration system (284,271 vehicles) with a retrofit-management system (48,716 cleaned visits), and compare strong tabular machine learning baselines with three LLM-based strategies on row-serialized inputs: embedding features (Amazon Titan), direct prompted classification (Claude Sonnet 4), and an ML+LLM stacking approach. Across binary occurrence prediction, 15-way retrofit-type classification, per-visit duration regression, and an aggregated monthly benchmark, classical tree ensembles remain the strongest standalone models. However, the LLM results reveal a consistent pattern: embeddings remain useful on tables (binary AUC = 0.982), direct prompting collapses once semantic signal is stripped by hashing (binary AUC = 0.500; multiclass weighted F1 = 0.018), and hybrid stacking yields the best manually built multiclass model (weighted F1 = 0.626). On the monthly benchmark, lag-based machine learning outperforms time-series foundation models, though Chronos-small remains competitive in zero-shot forecasting. The results suggest that on privacy-constrained industrial tables, LLMs are more effective as complementary components than as replacements for strong tabular baselines.

Problem

• tabular data
• limited semantics
• industrial retrofit prediction
• LLMs
• structured operational data

Innovation

• LLM on tabular data
• semantic-limited tables
• ML-LLM stacking
• industrial retrofit prediction
• zero-shot time-series forecasting