Related papers: Pile-Up Mitigation using Attention
One of the greatest impediments to extracting useful information from high luminosity hadron-collider data is radiation from secondary collisions (i.e. pileup) which can overlap with that of the primary interaction. In this paper we…
The pursuit of reducing the memory footprint of the self-attention mechanism in multi-headed self attention (MHA) spawned a rich portfolio of methods, e.g., group-query attention (GQA) and multi-head latent attention (MLA). The methods…
Efficient inference in Large Vision-Language Models is constrained by the high cost of processing thousands of visual tokens, yet it remains unclear which tokens and computations can be safely removed. While attention scores are commonly…
Due to the potentially severe consequences of coordinated cyber-physical attacks (CCPA), the design of defenses has gained significant attention. A popular approach is to eliminate the existence of attacks by either securing existing…
A novel deep neural network classifier, a ``Particle transformer'' (PaRT), is introduced for the identification of highly Lorentz-boosted resonances reconstructed as single, multipronged jets in measurements and searches performed by the…
Partial Automation (PA) with intelligent support systems has been introduced in industrial machinery and advanced automobiles to reduce the burden of long hours of human operation. Under PA, operators perform manual operations (providing…
Fast contact detection is crucial for safe human-robot collaboration. Observers based on proprioceptive information can be used for contact detection but have first-order error dynamics, which results in delays. Sensor fusion based on…
Most autonomous vehicles are equipped with LiDAR sensors and stereo cameras. The former is very accurate but generates sparse data, whereas the latter is dense, has rich texture and color information but difficult to extract robust 3D…
This work considers the problem of quickest detection of signals in a coupled system of N sensors, which receive continuous sequential observations from the environment. It is assumed that the signals, which are modeled a general Ito…
In urban cities, visual information on and along roadways is likely to distract drivers and lead to missing traffic signs and other accident-prone (AP) features. To avoid accidents due to missing these visual cues, this paper proposes a…
In complex engineering systems such as electro-thermal-fluid coupling, rapid and accurate prediction of multi-physics fields is essential for advanced applications like digital twins and real-time condition monitoring. Traditional numerical…
As large language models (LLMs) and visual language models (VLMs) grow in scale and application, attention mechanisms have become a central computational bottleneck due to their high memory and time complexity. While many efficient…
We study 100 images of early LHC collisions that were recorded by the ATLAS experiment and made public for outreach purposes, and extract the charged particle multiplicity as a function of momentum for proton-proton collisions at sqrt(s) =…
A significant fraction of pp collisions at the LHC will involve (quasi-real) photon interactions occurring at energies well beyond the electroweak energy scale. Hence, the LHC can to some extend be considered as a high-energy photon-photon…
Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency. However, conventional training-free structured pruning methods often employ a heuristic metric that…
Training causal transformers at extreme sequence lengths is bottlenecked by the quadratic time and memory of scaled dot-product attention (SDPA). In this work, we propose Lighthouse Attention, a training-only symmetrical selection-based…
Transformer-based object detectors often struggle with occlusions, fine-grained localization, and computational inefficiency caused by fixed queries and dense attention. We propose DAMM, Dual-stream Attention with Multi-Modal queries, a…
In this contribution, the production rates and the transverse momentum distributions of strange hadrons are reported as a function of charged particle multiplicity. In this analysis, the data collected in proton-proton collisions at…
Adversarial attacks against deep learning-based object detectors have been studied extensively in the past few years. Most of the attacks proposed have targeted the model's integrity (i.e., caused the model to make incorrect predictions),…
Large vision-language models (VLMs) are highly capable, yet often hallucinate by favoring textual prompts over visual evidence. We study this failure mode in a controlled object-counting setting, where the prompt overstates the number of…