相关论文: Patch Hierarchical Attention Transformer for Effic…

With the rise of Transformer models in NLP and CV domain, Multi-Head Attention has been proven to be a game-changer. However, its expensive computation poses challenges to the model throughput and efficiency, especially for the long…

While the Transformer architecture has become ubiquitous in the machine learning field, its adaptation to 3D shape recognition is non-trivial. Due to its quadratic computational complexity, the self-attention operator quickly becomes…

Jet tagging has become an essential tool for new physics searches at the high-energy frontier. For jets that contain energetic charged leptons we introduce Feature Extended Supervised Tagging (FEST) which, in addition to jet substructure,…

The maximum achievable performance of strange-jet tagging at hadron colliders and the loss in performance in different detector designs is estimated based on simulated truth jets from strange-quark and down-quark hadronisation. Both jet…

The increasing scale of deep learning models in high-energy physics (HEP) has posed challenges to their deployment on low-power, latency-sensitive platforms, such as FPGAs and ASICs used in trigger systems, as well as in offline data…

Measurements of jet substructure in ultra-relativistic heavy-ion collisions indicate that interactions with the quark-gluon plasma quench the jet showering process. Modern data-driven methods have shown promise in probing these…

We apply object detection techniques based on deep convolutional blocks to end-to-end jet identification and reconstruction tasks encountered at the CERN Large Hadron Collider (LHC). Collision events produced at the LHC and represented as…

A method is introduced for distinguishing top jets (boosted, hadronically decaying top quarks) from light quark and gluon jets using jet substructure. The procedure involves parsing the jet cluster to resolve its subjets, and then imposing…

Recent work has shown the potential of transformers for computer vision applications. An image is first partitioned into patches, which are then used as input tokens for the attention mechanism. Due to the expensive quadratic cost of the…

In this paper, we propose the Hierarchical Document Transformer (HDT), a novel sparse Transformer architecture tailored for structured hierarchical documents. Such documents are extremely important in numerous domains, including science,…

Top tagging is a recent approach to identifying boosted hadronic top quarks. It avoids reconstructing individual top decay products and instead uses a jet algorithm to reconstruct the entire top decay. Quite generally, geometrically large…

Multi-head self-attention is a distinctive feature extraction mechanism of vision transformers that computes pairwise relationships among all input patches, contributing significantly to their high performance. However, it is known to incur…

Transformer-based approaches have revolutionized image super-resolution by modeling long-range dependencies. However, the quadratic computational complexity of vanilla self-attention mechanisms poses significant challenges, often leading to…

This paper tackles the high computational/space complexity associated with Multi-Head Self-Attention (MHSA) in vanilla vision transformers. To this end, we propose Hierarchical MHSA (H-MHSA), a novel approach that computes self-attention in…

3D single object tracking is a key issue for robotics. In this paper, we propose a transformer module called Point-Track-Transformer (PTT) for point cloud-based 3D single object tracking. PTT module contains three blocks for feature…

In this study, we introduce the More-Interaction Particle Transformer (MIParT), a novel deep learning neural network designed for jet tagging. This framework incorporates our own design, the More-Interaction Attention (MIA) mechanism, which…

We study the performance of the Particle Transformer (ParT) for jet flavor tagging using ILD full simulation events (1M jets) as well as fast simulation samples (10M and 1M jets). We perform 3-category ($b/c/d$), 6-category ($b/c/d/u/s/g$),…

Jet flavor tagging is of utmost importance for unlocking the full physics potential of any future collider experiment. The performance of any jet flavor identification algorithm depends both on its underlying architecture and on the…

Attention mechanism is a significant part of Transformer models. It helps extract features from embedded vectors by adding global information and its expressivity has been proved to be powerful. Nevertheless, the quadratic complexity…

The task of reconstructing particles from low-level detector response data to predict the set of final state particles in collision events represents a set-to-set prediction task requiring the use of multiple features and their correlations…