Trackformers: in search of transformer-based particle tracking for the high-luminosity LHC era

2024-07-09

European Physical Journal C: Particles and Fields

AI Summary

To address the trajectory reconstruction bottleneck posed by massive detector hit data in the High-Luminosity LHC (HL-LHC) era, this work proposes a novel end-to-end particle tracking paradigm based on the Transformer architecture. Targeting the high computational cost of hit-to-track association in conventional methods, we pioneer the adaptation of large language model principles to high-energy physics tracking—introducing a dual-path architecture comprising “next-hit prediction” and “event-level full-track one-shot prediction.” We further demonstrate, for the first time, the feasibility and practicality of an encoder-classifier Transformer design for physics-based tracking. Evaluated on the lightweight TrackML dataset (5-level complexity) and the REDVID simulation framework, our approach achieves an optimal trade-off between accuracy and latency: the single-prediction Transformer attains state-of-the-art association accuracy in complex scenarios while meeting real-time trigger latency constraints.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Motion & Tracking
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

High-Energy Physics experiments are facing a multi-fold data increase with every new iteration. This is certainly the case for the upcoming High-Luminosity LHC upgrade. Such increased data processing requirements forces revisions to almost every step of the data processing pipeline. One such step in need of an overhaul is the task of particle track reconstruction, a.k.a., tracking. A Machine Learning-assisted solution is expected to provide significant improvements, since the most time-consuming step in tracking is the assignment of hits to particles or track candidates. This is the topic of this paper. We take inspiration from large language models. As such, we consider two approaches: the prediction of the next word in a sentence (next hit point in a track), as well as the one-shot prediction of all hits within an event. In an extensive design effort, we have experimented with three models based on the Transformer architecture and one model based on the U-Net architecture, performing track association predictions for collision event hit points. In our evaluation, we consider a spectrum of simple to complex representations of the problem, eliminating designs with lower metrics early on. We report extensive results, covering both prediction accuracy (score) and computational performance. We have made use of the REDVID simulation framework, as well as reductions applied to the TrackML data set, to compose five data sets from simple to complex, for our experiments. The results highlight distinct advantages among different designs in terms of prediction accuracy and computational performance, demonstrating the efficiency of our methodology. Most importantly, the results show the viability of a one-shot encoder-classifier based Transformer solution as a practical approach for the task of tracking.

Problem

• Developing Transformer-based models for particle track reconstruction
• Improving hit-to-track assignment accuracy in high-energy physics
• Evaluating computational performance of ML-assisted tracking solutions

Innovation

• Transformer-based next hit prediction for tracking
• One-shot encoder-classifier Transformer solution
• U-Net and Transformer models for track association