Related papers: Attention-Based Neural Network Emulators for Multi…

Modern electron tomography has progressed to higher resolution at lower doses by leveraging compressed sensing methods that minimize total variation (TV). However, these sparsity-emphasized reconstruction algorithms introduce tunable…

For many years now, understanding the brain mechanism has been a great research subject in many different fields. Brain signal processing and especially electroencephalogram (EEG) has recently known a growing interest both in academia and…

Recurrent Neural Networks were, until recently, one of the best ways to capture the timely dependencies in sequences. However, with the introduction of the Transformer, it has been proven that an architecture with only attention-mechanisms…

Deep learning surrogate models are being increasingly used in accelerating scientific simulations as a replacement for costly conventional numerical techniques. However, their use remains a significant challenge when dealing with real-world…

Quantum State Tomography (QST) is essential for characterizing and validating quantum systems, but its practical use is severely limited by the exponential growth of the Hilbert space and the number of measurements required for…

Transformer models typically calculate attention matrices using dot products, which have limitations when capturing nonlinear relationships between embedding vectors. We propose Neural Attention, a technique that replaces dot products with…

Characterization of quantum objects, being them states, processes, or measurements, complemented by previous knowledge about them is a valuable approach, especially as it leads to routine procedures for real-life components. To this end,…

Studying the impact of systematic effects, optimizing survey strategies, assessing tensions between different probes and exploring synergies of different data sets require a large number of simulated likelihood analyses, each of which cost…

Taking agent-based models (ABM) closer to the data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs combining supervised machine-learning and intelligent sampling to build a surrogate…

With the increasing computational demands of neural networks, many hardware accelerators for the neural networks have been proposed. Such existing neural network accelerators often focus on popular neural network types such as convolutional…

This paper presents a systematic literature review focusing on the application of machine learning techniques for deriving observational constraints in cosmology. The goal is to evaluate and synthesize existing research to identify…

In this work, quantum transformers are designed and analysed in detail by extending the state-of-the-art classical transformer neural network architectures known to be very performant in natural language processing and image analysis.…

State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion. Vaswani et al. (2017) propose a new architecture that avoids recurrence and convolution…

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and…

Bayesian parameter inference is an essential tool in modern cosmology, and typically requires the calculation of $10^5$--$10^6$ theoretical models for each inference of model parameters for a given dataset combination. Computing these…

Transformers are widely used deep learning architectures. Existing transformers are mostly designed for sequences (texts or time series), images or videos, and graphs. This paper proposes a novel transformer model for massive (up to a…

Neural networks (NNs) can achieved high performance in various fields such as computer vision, and natural language processing. However, deploying NNs in resource-constrained safety-critical systems has challenges due to uncertainty in the…

Much of recent Deep Reinforcement Learning success is owed to the neural architecture's potential to learn and use effective internal representations of the world. While many current algorithms access a simulator to train with a large…

Sequence-to-sequence (encoder-decoder) models with attention constitute a cornerstone of deep learning research, as they have enabled unprecedented sequential data modeling capabilities. This effectiveness largely stems from the capacity of…

The Bayesian uncertainty quantification technique has become well established in turbulence modeling over the past few years. However, it is computationally expensive to construct a globally accurate surrogate model for Bayesian inference…