TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction

2025-09-30

AI Summary

In the high-luminosity LHC era, the deluge of detector hit data poses critical accuracy and efficiency bottlenecks for traditional particle track reconstruction. To address this, we propose the first end-to-end, single-stage Transformer model operating directly on hit-level inputs. Our method employs an encoder-only architecture, incorporates a physics-informed attention mechanism that explicitly models physical correlations among hits, introduces a novel loss function tailored for track reconstruction, and supports joint training across multiple physics processes. Trained on both synthetic and real detector data, the model achieves significant improvements in track purity and efficiency while accelerating inference—demonstrating strong scalability and robustness. This work establishes the first purely deep learning–based track reconstruction paradigm for next-generation high-energy physics experiments, eliminating reliance on intermediate clustering or heuristic steps.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Motion & Tracking
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this study, we extend our earlier efforts by incorporating loss functions that account for inter-hit correlations, conducting detailed investigations into (various) Transformer attention mechanisms, and a study on the reconstruction of higher-level objects. Furthermore we discuss new datasets that allow the training on hit level for a range of physics processes. These developments collectively aim to boost both the accuracy, and potentially the efficiency of our tracking models, offering a robust solution to meet the demands of next-generation high-energy physics experiments.

Problem

• Enhancing transformer models for particle track reconstruction in high-energy physics
• Incorporating inter-hit correlations and studying transformer attention mechanisms
• Improving tracking accuracy and efficiency for next-generation physics experiments

Innovation

• Incorporated loss functions with inter-hit correlations
• Investigated various Transformer attention mechanisms
• Trained models on new datasets for multiple physics processes