David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM

2026-06-14

AI Summary

This study addresses the lack of systematic comparison among large language models (LLMs), Transformers, LSTMs, and simple baselines for next-activity prediction in process mining. The authors conduct a comprehensive evaluation across seven real-world event logs, assessing vocabulary-adapted LLMs, from-scratch trained Transformers, LLM-distilled Transformers, LSTMs, and a count-based argmax baseline. Their findings reveal that pretraining and model scale confer no significant performance advantage; remarkably, the simple argmax baseline matches or even surpasses billion-parameter LLMs on most datasets. These results challenge the presumed necessity of complex models for this task and underscore the critical influence of task-specific characteristics on model selection.

Abstract

Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.

Problem

• next activity prediction
• predictive process monitoring
• large language models
• Transformer
• LSTM

Innovation

• next activity prediction
• large language models
• Transformer
• LSTM
• argmax baseline