Related papers: Is attention all you need to solve the correlated …
We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schr\"odinger equation, the fundamental equation…
With the evolution of numerical methods, we are now aiming at not only qualitative understanding but also quantitative prediction and design of quantum many-body phenomena. As a novel numerical approach, machine learning techniques have…
Attention mechanisms are developing into a viable alternative to convolutional layers as elementary building block of NNs. Their main advantage is that they are not restricted to capture local dependencies in the input, but can draw…
The complexity of many-body quantum wave functions is a central aspect of several fields of physics and chemistry where non-perturbative interactions are prominent. Artificial neural networks (ANNs) have proven to be a flexible tool to…
Ab initio calculation of dielectric response with high-accuracy electronic structure methods is a long-standing problem, for which mean-field approaches are widely used and electron correlations are mostly treated via approximated…
The key to a Transformer model is the self-attention mechanism, which allows the model to analyze an entire sequence in a computationally efficient manner. Recent work has suggested the possibility that general attention mechanisms used by…
The challenge posed by the many-body problem in quantum physics originates from the difficulty of describing the non-trivial correlations encoded in the exponential complexity of the many-body wave function. Here we demonstrate that…
Attention mechanisms, and most prominently self-attention, are a powerful building block for processing not only text but also images. These provide a parameter efficient method for aggregating inputs. We focus on self-attention in vision…
Attention is a powerful component of modern neural networks across a wide variety of domains. However, despite its ubiquity in machine learning, there is a gap in our understanding of attention from a theoretical point of view. We propose a…
Object-based attention is a key component of the visual system, relevant for perception, learning, and memory. Neurons tuned to features of attended objects tend to be more active than those associated with non-attended objects. There is a…
Electron-electron interactions and correlations form the basis of difficulties encountered in the theoretical solution of problems dealing with multi-electron systems. Accurate treatment of the electron-electron problem is likely to unravel…
Learning from data has led to a paradigm shift in computational materials science. In particular, it has been shown that neural networks can learn the potential energy surface and interatomic forces through examples, thus bypassing the…
Convolutional neural networks have allowed remarkable advances in single image super-resolution (SISR) over the last decade. Among recent advances in SISR, attention mechanisms are crucial for high-performance SR models. However, the…
The quantum mechanical many-body problem is rarely analytically solvable. One notable exception is the case of two electrons interacting via a Coulomb potential in a uniform magnetic field. The motion is confined to a two-dimensional plane,…
The self-attention mechanism, at the heart of the Transformer model, is able to effectively model pairwise interactions between tokens. However, numerous recent works have shown that it is unable to perform basic tasks involving detecting…
Neural networks (NNs) have great potential in solving the ground state of various many-body problems. However, several key challenges remain to be overcome before NNs can tackle problems and system sizes inaccessible with more established…
Point defects are of interest for many applications, from quantum sensing to modifying bulk properties of materials. Because of their localized orbitals, the electronic states are often strongly correlated, which has led to a proliferation…
Variational methods have proven to be excellent tools to approximate ground states of complex many body Hamiltonians. Generic tools like neural networks are extremely powerful, but their parameters are not necessarily physically motivated.…
The development of numerical methods capable of simulating realistic materials with strongly correlated electrons, with controllable errors, is a central challenge in quantum many-body physics. Here we describe how a hybrid between…
Over the past years, machine learning has emerged as a powerful computational tool to tackle complex problems over a broad range of scientific disciplines. In particular, artificial neural networks have been successfully deployed to…