Related papers: Interpreting Transformers for Jet Tagging
Recent jet and jet substructure measurements at the LHC, and of machine-learning-based tagging techniques are presented using proton-proton collision data collected by the ATLAS and CMS experiments at CERN's Large Hadron Collider. These…
The increasing scale of deep learning models in high-energy physics (HEP) has posed challenges to their deployment on low-power, latency-sensitive platforms, such as FPGAs and ASICs used in trigger systems, as well as in offline data…
A deep-learning approach based on the transformer architecture is developed to distinguish between jets originating from quarks and gluons. The algorithm operates on jets with transverse momentum $p_{\text{T}} > 20$ and pseudorapidity…
Jet classification in high-energy particle physics is important for understanding fundamental interactions and probing phenomena beyond the Standard Model. Jets originate from the fragmentation and hadronization of quarks and gluons, and…
An attention matrix of a transformer self-attention sublayer can provably be decomposed into two components and only one of them (effective attention) contributes to the model output. This leads us to ask whether visualizing effective…
Convolutional neural networks are basic structures using jet images as input for the jet tagging problems. However, what they have learned during the training process is always difficult to understand just through feature maps. Inspired by…
The application of machine learning (ML) in high energy physics (HEP), specifically in heavy-flavor jet tagging at Large Hadron Collider (LHC) experiments, has experienced remarkable growth and innovation in the past decade. This review…
Jet flavour tagging enables the identification of jets originating from heavy-flavour quarks in proton-proton collisions at the Large Hadron Collider, playing a critical role in its physics programmes. This paper presents GN2, a…
Designing a single neural network architecture that performs competitively across a range of molecule property prediction tasks remains largely an open challenge, and its solution may unlock a widespread use of deep learning in the drug…
A tagging algorithm to identify jets that are significantly displaced from the proton-proton (pp) collision region in the CMS detector at the LHC is presented. Displaced jets can arise from the decays of long-lived particles (LLPs), which…
The point cloud learning community witnesses a modeling shift from CNNs to Transformers, where pure Transformer architectures have achieved top accuracy on the major learning benchmarks. However, existing point Transformers are…
The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions…
Jet flavor tagging plays an important role in precise Standard Model measurement enabling the extraction of mass dependence in jet-quark interaction and quark-gluon plasma (QGP) interactions. They also enable inferring the nature of…
We present the first sub-microsecond transformer implementation on an FPGA achieving competitive performance for state-of-the-art high-energy physics benchmarks. Transformers have shown exceptional performance on multiple tasks in modern…
The separation of $b$-quark initiated jets from those coming from lighter quark flavors ($b$-tagging) is a fundamental tool for the ATLAS physics program at the CERN Large Hadron Collider. The most powerful $b$-tagging algorithms combine…
Jet modification in heavy-ion collisions provides microscopic access to the properties of the quark-gluon plasma. However, conventional approaches based on traditional global observables, such as \(R_{AA}\), capture limited information…
Transformer-based models are popularly used in natural language processing (NLP). Its core component, self-attention, has aroused widespread interest. To understand the self-attention mechanism, a direct method is to visualize the attention…
We deploy an advanced Machine Learning (ML) environment, leveraging a multi-scale cross-attention encoder for event classification, towards the identification of the $gg\to H\to hh\to b\bar b b\bar b$ process at the High Luminosity Large…
The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model…
In this paper, we introduce \texttt{IAFormer}, a novel Transformer-based architecture that efficiently integrates pairwise particle interactions through a dynamic sparse attention mechanism. \texttt{IAFormer} has two new mechanisms within…