Related papers: Pile-Up Mitigation using Attention

Two-particle angular correlations are studied in $pp$ collisions at $\sqrt{s}=\,13\,$TeV, collected with the LHCb detector at the LHC. The LHCb detector provides measurement in the very forward region, $2 < \eta < 5$. This region is…

One of the major challenges for the LHC will be to extract precise information from hadronic final states in the presence of the large number of additional soft pp collisions, pileup, that occur simultaneously with any hard interaction in…

A particle flow event-reconstruction algorithm has been successfully deployed in the CMS experiment and is nowadays used by most of the analyses. It aims at identifying and reconstructing individually each particle arising from the LHC…

Recently many effective attention modules are proposed to boot the model performance by exploiting the internal information of convolutional neural networks in computer vision. In general, many previous works ignore considering the design…

Protons and heavy-ion beams at unprecedented energies are brought into collisions in the CERN Large Hadron Collider for high-energy experiments. The LHC multi-stage collimation system is designed to provide protection against regular and…

Attention serves as the fundamental mechanism for long-context modeling in large language models (LLMs), yet dense attention becomes structurally prohibitive for long sequences due to its quadratic complexity. Consequently, sparse attention…

The cleanest way to discover a new particle is generally the "bump-hunt" methodology: looking for a localised excess in a mass (or related) distribution. However, if the mass of the particle being discovered is not known the procedure of…

Transformer self-attention computes pairwise token interactions, yet protein sequence to phenotype relationships often involve cooperative dependencies among three or more residues that dot product attention does not capture explicitly. We…

As part of the Phase II upgrade program, the Compact Muon Solenoid (CMS) detector will incorporate a new timing layer designed to measure minimum ionizing particles (MIPs) with a time resolution of $\sim$30 ps. Precision timing will…

At the start of Run 2 in 2015, the LHC delivered proton-proton collisions at a center-of-mass energy of 13 TeV. During Run 2 (years 2015-2018) the LHC eventually reached a luminosity of 2.1 $\times$ 10$^{34}$ cm$^{-2}$ s$^{-1}$, almost…

Scaling pre-trained language models has resulted in large performance gains in various natural language processing tasks but comes with a large cost in memory requirements. Inspired by the position embeddings in transformers, we aim to…

Computational workloads composing traditional transformer models are starkly bifurcated. Multi-Head Attention (MHA) and Grouped-Query Attention are memory-bound due to low arithmetic intensity, while FeedForward Networks are compute-bound.…

We present a machine learning (ML) method to calibrate hadronic jet energy in real-time trigger systems of the High-Luminosity Large Hadron Collider (HL-LHC) using an efficient implementation on field programmable gate arrays (FPGA).…

Data scouting, introduced by CMS in 2011, is the use of specialized data streams based on reduced event content, enabling LHC experiments to record unprecedented numbers of proton-proton collision events that would otherwise be rejected by…

The prospect of pileup induced backgrounds at the High Luminosity LHC (HL-LHC) has stimulated intense interest in technology for charged particle timing at high rates. In contrast to the role of timing for particle identification, which has…

The vanilla self-attention mechanism in Transformers can be viewed as a two-layer fast-weight MLP, whose weights are dynamically induced by inputs and whose hidden dimension is equal to the sequence length $N$. As the context extends, the…

Precision timing has played a critical role in high-energy physics experiments, particularly for particle identification and the suppression of pileup under the challenging conditions expected at future colliders like the High-Luminosity…

Due to a limited bandwidth and a large proton-proton interaction cross-section relative to the rate of interesting physics processes, most events produced at the Large Hadron Collider (LHC) are discarded in real time. A sophisticated…

This paper presents LAPA (Look Around and Pay Attention), a novel end-to-end transformer-based architecture for multi-camera point tracking that integrates appearance-based matching with geometric constraints. Traditional pipelines decouple…

Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view…