Related papers: Is attention all you need to solve the correlated …
Deep metric learning aims to learn an embedding function, modeled as deep neural network. This embedding function usually puts semantically similar images close while dissimilar images far from each other in the learned embedding space.…
We propose an efficient interactive method for multi-head self-attention via decomposition. For existing methods using multi-head self-attention, the attention operation of each head is computed independently. However, we show that the…
To support humans in their daily lives, robots are required to autonomously learn, adapt to objects and environments, and perform the appropriate actions. We tackled on the task of cooking scrambled eggs using real ingredients, in which the…
Recent quantum algorithms pertaining to electronic structure theory primarily focus on threshold-based dynamic construction of ansatz by selectively including important many-body operators. These methods can be made systematically more…
The Transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the Vision Transformer (ViT) architecture. The key feature is the ability…
A large collaboration carefully benchmarks 20 first principles many-body electronic structure methods on a test set of 7 transition metal atoms, and their ions and monoxides. Good agreement is attained between the 3 systematically converged…
Objects are entities we act upon, where the functionality of an object is determined by how we interact with it. In this work we propose a Dual Attention Network model which reasons about human-object interactions. The dual-attentional…
We present a method which computes many-electron energies and eigenfunctions by a full configuration interaction which uses a basis of atomistic tight-binding wave functions. This approach captures electron correlation as well as atomistic…
Attention mechanism has played critical roles in various state-of-the-art NLP models such as Transformer and BERT. It can be formulated as a ternary function that maps the input queries, keys and values into an output by using a summation…
The many-body Coulomb repulsive energy of strictly correlated electrons provides direct information of the exact Hohenberg-Kohn exchange-correlation functional in the strong interaction limit. Until now the treatment of strictly correlated…
The spreading of attention has been proposed as a mechanism for how humans group features to segment objects. However, such a mechanism has not yet been implemented and tested in naturalistic images. Here, we leverage the feature maps from…
Analysis of quantum dot structures is a current topic with important applications in solid-state digital logic design, quantum information technology and quantum optics. In this work, we show a variational formulation for the solution of…
We outline a rigorous method which can be used to solve the many-body Schroedinger equation for a Coulomb interacting electronic system in an external classical magnetic field as well as a quantized electromagnetic field. Effects of the…
We consider the problem of predicting edges in a graph from node attributes in an e-commerce setting. Specifically, given nodes labelled with search query text, we want to predict links to related queries that share products. Experiments…
We introduce a new architecture for unsupervised object-centric representation learning and multi-object detection and segmentation, which uses a translation-equivariant attention mechanism to predict the coordinates of the objects present…
Attention mechanisms in biological perception are thought to select subsets of perceptual information for more sophisticated processing which would be prohibitive to perform on all sensory inputs. In computer vision, however, there has been…
Transformers have demonstrated remarkable performance across diverse domains. The key component of Transformers is self-attention, which learns the relationship between any two tokens in the input sequence. Recent studies have revealed that…
Attention is a powerful and ubiquitous mechanism for allowing neural models to focus on particular salient pieces of information by taking their weighted average when making predictions. In particular, multi-headed attention is a driving…
In this paper we present a deeper analysis than has previously been carried out of a selective attention problem, and the evolution of continuous-time recurrent neural networks to solve it. We show that the task has a rich structure, and…
Neutral molecules with sufficiently large dipole moments can bind electrons in diffuse nonvalence orbitals with most of their charge density far from the nuclei, forming so-called dipole-bound anions. Because long-range correlation effects…