LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

2026-08-24

AI Summary

研究基于大语言模型的预测代理，通过结合语言推理、时间数据等方法解决未来或未观测目标的预测问题，并探讨了训练、评估及应用。

Abstract

Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external tools, and iterative prediction. We investigate LLM-based forecasting agents, meaning systems in which a language model contributes to a scored prediction about a future or currently unobserved target. We organize architectures into three groups. Standalone LLM workflows operate on encoded time series or event context. Tool- and retrieval-augmented agents incorporate external evidence. Hybrid systems pair LLMs with statistical or foundation models. We then review training methods and evaluation protocols. We examine negative as well as positive evidence, including sensitivity to small input perturbations, ablations in which the LLM component does not improve accuracy, and benchmark gains that may reflect contamination instead of temporal reasoning. We cover applications in finance, weather, health, energy, and operations, and we summarize the benchmarks and datasets used for evaluation. The evidence indicates that measurement is a central limitation. Future work requires calibration under distribution shift, contamination-resistant live evaluation, explicit reporting of cost and accuracy together, and methods for handling feedback between deployed forecasts and the outcomes being forecast.

Problem

• Large Language Models
• Forecasting
• Temporal Data
• Evaluation Protocols
• Distribution Shift

Innovation

• LLM-based forecasting
• Temporal reasoning
• Hybrid systems
• Distribution shift
• Live evaluation