Related papers: Reconstructing hadronically decaying tau leptons w…
Tensor decomposition is one of the well-known approaches to reduce the latency time and number of parameters of a pre-trained model. However, in this paper, we propose an approach to use tensor decomposition to reduce training time of…
Despite the superior performance, it is challenging to deploy foundation models or large language models (LLMs) due to their massive parameters and computations. While pruning is a promising technique to reduce model size and accelerate the…
This note describes the validation of a new form of re-interpretation material provided by an ATLAS search for hadronically-decaying neutral long-lived particles in association with jets or leptons, using the full Run-2 dataset. This…
Despite the impressive advancements achieved using deep-learning for functional brain activity analysis, the heterogeneity of functional patterns and scarcity of imaging data still pose challenges in tasks such as prediction of future onset…
Applying a pre-trained large model to downstream tasks is prohibitive under resource-constrained conditions. Recent dominant approaches for addressing efficiency issues involve adding a few learnable parameters to the fixed backbone model.…
PDE foundation models are typically pretrained on large, diverse corpora of PDE datasets and can be adapted to new settings with limited task-specific data. However, most downstream evaluations focus on forward problems, such as…
Accurate reconstruction of the mass of a resonance decaying to a pair of $\tau$ leptons is challenging because of the presence of multiple neutrinos from $\tau$ decays. The existing methods rely on either a partially reconstructed mass,…
We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific tasks, which limits their adaptability across diverse applications. In…
Geometric machine learning models such as graph neural networks have achieved remarkable success in recent years in chemical and materials science research for applications such as high-throughput virtual screening and atomistic…
Processes with $\tau$-leptons in the final state are important for Standard Model measurements and searches for physics beyond the Standard Model. The ATLAS experiment at the Large Hadron Collider observes $\tau$-leptons produced in…
In the particle-flow approach information from all available sub-detector systems is combined to reconstruct all stable particles. The global event reconstruction has been shown to improve, in particular, the resolution of jet energy and…
Existing foundation models (FMs) in the medical domain often require extensive fine-tuning or rely on training resource-intensive decoders, while many existing encoders are pretrained with objectives biased toward specific tasks. This…
Machine-learning models in high-energy physics are often trained on simulated data, where fully simulated samples are computationally expensive while fast simulation provides large statistics at reduced realism. In this work, we…
Recent advances in AI weather forecasting have led to the emergence of so-called "foundation models", typically defined by expensive pretraining and minimal fine-tuning for downstream tasks. However, in the natural sciences, a desirable…
The reconstruction of the invariant mass of $\tau$ lepton pairs is important for analyses containing Higgs and Z bosons decaying to $\tau^{+}\tau^{-}$, but highly challenging due to the neutrinos from the $\tau$ lepton decays, which cannot…
Foundation models serve as the backbone for numerous specialized models developed through fine-tuning. However, when the underlying pretrained model is updated or retrained (e.g., on larger and more curated datasets), the fine-tuned model…
While machine learning is widely used to optimize wireless networks, training a separate model for each task in communication and localization is becoming increasingly unsustainable due to the significant costs associated with training and…
Machine-learned interatomic potentials are revolutionising atomistic materials simulations by providing accurate and scalable predictions within the scope covered by the training data. However, generation of an accurate and robust training…
This paper describes the algorithms used by the CMS experiment to reconstruct and identify tau to hadrons + tau neutrino decays during Run 1 of the LHC. The performance of the algorithms is studied in proton-proton collisions recorded at a…
Adapting large pre-trained models to unseen tasks under tight data and compute budgets remains challenging. Meta-learning approaches explicitly learn good initializations, but they require an additional meta-training phase over many tasks,…