相关论文: Learning the language of QCD jets with transformer…

Jets are suppressed and modified in heavy ion collisions, which serve as powerful probes to the properties of the quark-gluon plasma (QGP). Attributed to the abundant information carried by the jet constituents and reconstructed…

The Transformer model is widely used in natural language processing for sentence representation. However, the previous Transformer-based models focus on function words that have limited meaning in most cases and could merely extract…

The classification of events involving jets as signal-like or background-like can depend strongly on the jet algorithm used and its parameters. This is partly due to the fact that standard jet algorithms yield a single partition of the…

A novel deep neural network classifier, a ``Particle transformer'' (PaRT), is introduced for the identification of highly Lorentz-boosted resonances reconstructed as single, multipronged jets in measurements and searches performed by the…

Fine-tuned transformer models have shown superior performances in many natural language tasks. However, the large model size prohibits deploying high-performance transformer models on resource-constrained devices. This paper proposes a…

Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and…

The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous,…

We discuss jet substructure in recombination algorithms for QCD jets and single jets from heavy particle decays. We demonstrate that the jet algorithm can introduce significant systematic effects into the substructure. By characterizing…

The study of the substructure of collimated particles from quarks and gluons, or jets, has the promise to reveal the details how color charges interact with the QCD plasma medium created in colliders such as RHIC and the LHC. Traditional…

The transformers have achieved significant accomplishments in the natural language processing as its outstanding parallel processing capabilities and highly flexible attention mechanism. In addition, increasing studies based on transformers…

When trained on language data, do transformers learn some arbitrary computation that utilizes the full capacity of the architecture or do they learn a simpler, tree-like computation, hypothesized to underlie compositional meaning systems…

Predicting upcoming events is critical to our ability to interact with our environment. Transformer models, trained on next-word prediction, appear to construct representations of linguistic input that can support diverse downstream tasks.…

Boosted resonances is a highly probable and enthusiastic scenario in any process probing the electroweak scale. Such objects when decaying into jets can easily blend with the cornucopia of jets from hard relative light QCD states. We review…

Understanding how Transformer-based Language Models (LMs) learn and recall information is a key goal of the deep learning community. Recent interpretability methods project weights and hidden states obtained from the forward pass to the…

Jets with transverse energy of few TeV are becoming now common in LHC data. Most of these jets are produced by QCD processes and some from the collimated decay of highly boosted objects like W, Z, H0 and top-quark. The study of such QCD…

Many analyses at the collider utilize the hadronic jets that are the footprints of QCD partons. These are used both to study the QCD processes themselves and increasingly as tools to study other physics, for example top mass reconstruction.…

We develop a generative neural network for the generation of sparse data in particle physics using a permutation-invariant and physics-informed loss function. The input dataset used in this study consists of the particle constituents of…

Transformers are neural networks that revolutionized natural language processing and machine learning. They process sequences of inputs, like words, using a mechanism called self-attention, which is trained via masked language modeling…

Transformers have supplanted recurrent models in a large number of NLP tasks. However, the differences in their abilities to model different syntactic properties remain largely unknown. Past works suggest that LSTMs generalize very well on…

One of the most recent and fascinating breakthroughs in artificial intelligence is ChatGPT, a chatbot which can simulate human conversation. ChatGPT is an instance of GPT4, which is a language model based on generative gredictive…