Related papers: Is attention all you need to solve the correlated …
In Click-through rate (CTR) prediction models, a user's interest is usually represented as a fixed-length vector based on her history behaviors. Recently, several methods are proposed to learn an attentive weight for each user behavior and…
In humans, Attention is a core property of all perceptual and cognitive operations. Given our limited ability to process competing sources, attention mechanisms select, modulate, and focus on the information most relevant to behavior. For…
Ab initio calculations play an essential role in our fundamental understanding of quantum many-body systems across many subfields, from strongly correlated fermions to quantum chemistry and from atomic and molecular systems to nuclear…
Variational quantum algorithms have been a promising candidate to utilize near-term quantum devices to solve real-world problems. The powerfulness of variational quantum algorithms is ultimately determined by the expressiveness of the…
A quantum machine consisting of interacting linear clusters of atoms is proposed for the 3SAT problem. Each cluster with two relevant states of collective motion can be used to register a Boolean variable. Given any 3SAT Boolean formula the…
Humans can effectively find salient regions in complex scenes. Self-attention mechanisms were introduced into Computer Vision (CV) to achieve this. Attention Augmented Convolutional Network (AANet) is a mixture of convolution and…
Human-like attention as a supervisory signal to guide neural attention has shown significant promise but is currently limited to uni-modal integration - even for inherently multimodal tasks such as visual question answering (VQA). We…
Neural language models are becoming the prevailing methodology for the tasks of query answering, text classification, disambiguation, completion and translation. Commonly comprised of hundreds of millions of parameters, these neural network…
Self-attention mechanisms model long-range context by using pairwise attention between all input tokens. In doing so, they assume a fixed attention granularity defined by the individual tokens (e.g., text characters or image pixels), which…
The attention module is the key component in Transformers. While the global attention mechanism offers high expressiveness, its excessive computational cost restricts its applicability in various scenarios. In this paper, we propose a novel…
Attention layers are an integral part of modern end-to-end automatic speech recognition systems, for instance as part of the Transformer or Conformer architecture. Attention is typically multi-headed, where each head has an independent set…
Processing spatial data is a key component in many learning tasks for autonomous driving such as motion forecasting, multi-agent simulation, and planning. Prior works have demonstrated the value in using SE(2) invariant network…
We study the optimal focusing of two-level atoms with a near resonant standing wave light, using both classical and quantum treatments of the problem. Operation of the focusing setup is considered as a nonlinear spatial squeezing of atoms…
Accurately solving the Schr\"odinger equation for intricate systems remains a prominent challenge in physical sciences. A paradigm-shifting approach to address this challenge involves the application of artificial intelligence techniques.…
Electronic nearsightedness is one of the fundamental principles governing the behavior of condensed matter and supporting its description in terms of local entities such as chemical bonds. Locality also underlies the tremendous success of…
In this paper we aim to answer questions based on images when provided with a dataset of question-answer pairs for a number of images during training. A number of methods have focused on solving this problem by using image based attention.…
In this paper, we solve quantum many-body problem by propagating ensembles of trajectories and guiding waves in physical space. We introduce the 'effective potential' correction within the recently proposed time-dependent quantum Monte…
The information available to robots in real tasks is widely distributed both in time and space, requiring the agent to search for relevant data. In humans, that face the same problem when sounds, images and smells are presented to their…
In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention…
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale inference. This has motivated a rapidly growing body of work…