Related papers: Is attention all you need to solve the correlated …
Variational ab-initio methods in quantum chemistry stand out among other methods in providing direct access to the wave function. This allows in principle straightforward extraction of any other observable of interest, besides the energy,…
In this paper we propose an ab initio method to solve quantum many-body problems of molecular dynamics where both the electronic and the nuclear degrees are represented by ensembles of trajectories and guiding waves in physical space. Both…
Multi-head, key-value attention is the backbone of the widely successful Transformer model and its variants. This attention mechanism uses multiple parallel key-value attention blocks (called heads), each performing two fundamental…
Accurate solution of the many-electron problem including correlations remains intractable except for few-electron systems. Describing interacting electrons as a superposition of independent electron configurations results in an apparent…
We introduce Attention Free Transformer (AFT), an efficient variant of Transformers that eliminates the need for dot product self attention. In an AFT layer, the key and value are first combined with a set of learned position biases, the…
Designing a single neural network architecture that performs competitively across a range of molecule property prediction tasks remains largely an open challenge, and its solution may unlock a widespread use of deep learning in the drug…
We discuss differences and similarities between variational Monte Carlo approaches that use conventional and artificial neural network parameterizations of the ground-state wave function for systems of fermions. We focus on a relatively…
Transformers are built upon multi-head scaled dot-product attention and positional encoding, which aim to learn the feature representations and token dependencies. In this work, we focus on enhancing the distinctive representation by…
Recently, radical progress in machine learning (ML) has revolutionized computational materials science, enabling unprecedentedly rapid materials discovery and property prediction, but the quantum many-body problem -- which is the key to…
Highly excited states of quantum many-body systems are central objects in the study of quantum dynamics and thermalization that challenge classical computational methods due to their volume-law entanglement content. In this work, we explore…
We present JefiAtten, a novel neural network model employing the attention mechanism to solve Maxwell's equations efficiently. JefiAtten uses self-attention and cross-attention modules to understand the interplay between charge density,…
In the realm of deep learning, the self-attention mechanism has substantiated its pivotal role across a myriad of tasks, encompassing natural language processing and computer vision. Despite achieving success across diverse applications,…
Semantic correspondence is the problem of establishing correspondences across images depicting different instances of the same object or scene class. One of recent approaches to this problem is to estimate parameters of a global…
Attention mechanisms have been boosting the performance of deep learning models on a wide range of applications, ranging from speech understanding to program induction. However, despite experiments from psychology which suggest that…
Human attention modelling has proven, in recent years, to be particularly useful not only for understanding the cognitive processes underlying visual exploration, but also for providing support to artificial intelligence models that aim to…
Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing. In addition to yielding gains in predictive accuracy, attention weights are often claimed to confer interpretability, purportedly…
Predicting relaxed atomic structures of chemically complex materials remains a major computational challenge, particularly for high-entropy systems where traditional first-principles methods become prohibitively expensive. We introduce the…
While self-attention mechanism has shown promising results for many vision tasks, it only considers the current features at a time. We show that such a manner cannot take full advantage of the attention mechanism. In this paper, we present…
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set,…
We consider the exact solution of a model of correlated particles, which is presented as a system of interacting XX and Fateev-Zamolodchikov chains. This model can also be considered as a generalization of the multiband anisotropic $t-J$…