Neural Scaling Laws for Jet Generation

2026-05-27

AI Summary

This work investigates the existence of neural scaling laws in particle jet generation tasks to inform the training and physics performance evaluation of large-scale generative models. Through systematic analysis of how model size, dataset volume, and computational budget affect generation quality, the study reveals—for the first time—a logarithmic scaling relationship between model scale and both validation loss and physical observables. It further introduces the concept of a “learnable window” to explain the rapid saturation observed in data and compute scaling effects. Using autoregressive generative models, next-token prediction loss, and sliced Wasserstein distance to assess fidelity to physical distributions, the experiments demonstrate a strong correlation between validation loss and physical performance, thereby providing both theoretical grounding and practical guidance for generative modeling in high-energy physics.

Abstract

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Problem

• neural scaling laws
• jet generation
• foundation models
• autoregressive modeling
• particle physics

Innovation

• neural scaling laws
• jet generation
• sliced Wasserstein distance
• autoregressive modeling
• learnable window