相关论文: Learning the language of QCD jets with transformer…

In high-energy particle collisions, the reconstruction of secondary vertices from heavy-flavour hadron decays is crucial for identifying and studying jets initiated by $b$- or $c$-quarks. Traditional methods, while effective, require…

Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and materials discovery. Self-supervised pretraining of transformer models requires large-scale…

The suppression of the yield of high transverse momentum $p_{T}$ hadrons in heavy-ion collisions, referred to as "jet-quenching", has now developed into a comprehensive science. Jets are now used as probes of a variety of properties of the…

Despite their empirical success, the internal mechanism by which transformer models align tokens during language processing remains poorly understood. This paper provides a mechanistic and theoretical explanation of token alignment in LLMs.…

In the past years significant progress has been made toward achieving a quantitative understanding of jets and their substructure in high-energy proton-proton collisions from first principles in QCD. Precise measurements have become…

Jets at high energy colliders are complicated objects to identify. Even if jets are widely separated, there is no reason for jets to have the same size. A single reconstruction, or interpretation, of each event can only extract a limited…

Sentence embeddings can be decoded to give approximations of the original texts used to create them. We explore this effect in the context of text simplification, demonstrating that reconstructed text embeddings preserve complexity levels.…

The transformer is a neural network component that can be used to learn useful representations of sequences or sets of data-points. The transformer has driven recent advances in natural language processing, computer vision, and…

Copious production of very energetic jets is expected at the LHC due to the large increase in collision energy. Jet reconstruction at these high center of mass energies will provide crucial leverage to map out the QCD evolution of parton…

In this article, we review recent machine learning methods used in challenging particle identification of heavy-boosted particles at high-energy colliders. Our primary focus is on attention-based Transformer networks. We report the…

New particles beyond the Standard Model might be produced with a very high boost, for instance if they result from the decay of a heavier particle. If the former decay hadronically, then their signature is a single massive fat jet which is…

Transformers underlie almost all state-of-the-art language models in computational linguistics, yet their cognitive adequacy as models of human sentence processing remains disputed. In this work, we use a surprisal-based linking mechanism…

Language transformations are algorithms that take a language specification in input, and return the language specification modified. Language transformations are useful for automatically adding features such as subtyping to programming…

We apply a Dense Neural Network (DNN) approach to reconstruct jet momentum within a quark-gluon plasma (QGP) background, using simulated data from PYTHIA and Linear Boltzmann Transport (LBT) Models for comparative analysis. We find that…

The underlying structure of natural language is hierarchical; words combine into phrases, which in turn form clauses. An awareness of this hierarchical structure can aid machine learning models in performing many linguistic tasks. However,…

Transformers are a popular choice for classification tasks and as backbones for object detection tasks. However, their high latency brings challenges in their adaptation to lightweight object detection systems. We present an approximation…

Large language models exhibit sophisticated capabilities, yet understanding how they work internally remains a central challenge. A fundamental obstacle is that training selects for behavior, not circuitry, so many weight configurations can…

Jets with a large radius $R\gtrsim 1$ and grooming algorithms are widely used to fully capture the decay products of boosted heavy particles at the Large Hadron Collider (LHC). Unlike most discriminating variables used in such studies, the…

Machine-learning models in high-energy physics are often trained on simulated data, where fully simulated samples are computationally expensive while fast simulation provides large statistics at reduced realism. In this work, we…

Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems. This approach stands in contrast to autoencoders, also trained on raw text, but with the…