Related papers: Is attention all you need to solve the correlated …

Self-attention networks (SANs) have drawn increasing interest due to their high parallelization in computation and flexibility in modeling dependencies. SANs can be further enhanced with multi-head attention by allowing the model to attend…

Modern computational organic chemistry is becoming increasingly data-driven. There remain a large number of important unsolved problems in this area such as product prediction given reactants, drug discovery, and metric-optimized molecule…

Humans can selectively focus on different information based on different tasks requirements, other people's abilities and availability. Therefore, they can adapt quickly to a completely different and complex environments. If, like people,…

Electron correlation effects are essential for an accurate ab initio description of molecules. A quantitative a priori knowledge of the single- or multi-reference nature of electronic structures as well as of the dominant contributions to…

Multivariate time series anomaly detection (MTAD) plays a vital role in a wide variety of real-world application domains. Over the past few years, MTAD has attracted rapidly increasing attention from both academia and industry. Many deep…

We calculate analytically the exact dynamical response of a droplet of N interacting electrons in a quantum dot with an arbitrarily time-dependent parabolic confinement potential \omega(t) and a perpendicular magnetic field. We find that,…

Quantum convolutional neural networks (QCNNs) have gathered attention as one of the most promising algorithms for quantum machine learning. Reduction in the cost of training as well as improvement in performance is required for practical…

Attentional Neural Network is a new framework that integrates top-down cognitive bias and bottom-up feature extraction in one coherent architecture. The top-down influence is especially effective when dealing with high noise or difficult…

Machine learning approaches have recently been applied to the study of various problems in physics. Most of the studies are focused on interpreting the data generated by conventional numerical methods or an existing database. An interesting…

Convolutional Neural Networks have played a significant role in various medical imaging tasks like classification and segmentation. They provide state-of-the-art performance compared to classical image processing algorithms. However, the…

The self-attention mechanism has significantly advanced the field of natural language processing, facilitating the development of advanced language-learning machines. Although its utility is widely acknowledged, the precise mechanisms of…

The resources required to solve the general interacting quantum N-body problem scale exponentially with N, making the solution of this problem very difficult when N is large. In a previous series of papers we develop an approach for a…

A central problem in quantum mechanics involves solving the Electronic Schrodinger Equation for a molecule or material. The Variational Monte Carlo approach to this problem approximates a particular variational objective via sampling, and…

Transformer-based models have emerged as a leading architecture for natural language processing, natural language generation, and image generation tasks. A fundamental element of the transformer architecture is self-attention, which allows…

[New and updated results were published in Nature Chemistry, doi:10.1038/s41557-020-0544-y.] The electronic Schr\"odinger equation describes fundamental properties of molecules and materials, but can only be solved analytically for the…

Latest development of neural models has connected the encoder and decoder through a self-attention mechanism. In particular, Transformer, which is solely based on self-attention, has led to breakthroughs in Natural Language Processing (NLP)…

We investigate attention as the active pursuit of useful information. This contrasts with attention as a mechanism for the attenuation of irrelevant information. We also consider the role of short-term memory, whose use is critical to any…

The quadratic complexity of standard attention mechanisms poses a significant scalability bottleneck for large language models (LLMs) in long-context scenarios. While hybrid attention strategies that combine sparse and full attention within…

A two-particle self-consistency is rarely part of mean-field theories. It is, however, essential for avoiding spurious critical transitions and unphysical behavior. We present a general scheme for constructing analytically controllable…

The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-token interactions),…