Related papers: Self-attention enabled quantum path analysis of hi…
Vision Transformers are very popular nowadays due to their state-of-the-art performance in several computer vision tasks, such as image classification and action recognition. Although their performance has been greatly enhanced through…
We present a quantum optical generalization of the quantum-matter Lewenstein model of high-harmonic generation (HHG) in gases that contains two channels corresponding to the inter- and intraband HHG in solids. Both channels can be presented…
This study aims to enhance the quality of music generation using Transformers by incorporating meta-information. While Transformer-based approaches are effective at capturing long-term dependencies in musical compositions, the music they…
Uncovering hidden graph structures underlying real-world data is a critical challenge with broad applications across scientific domains. Recently, transformer-based models leveraging the attention mechanism have demonstrated strong…
The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention…
Linear attention mechanisms have emerged as efficient alternatives to full self-attention in Graph Transformers, offering linear time complexity. However, existing linear attention models often suffer from a significant drop in…
Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various…
Attention mechanism has gained huge popularity due to its effectiveness in achieving high accuracy in different domains. But attention is opportunistic and is not justified by the content or usability of the content. Transformer like…
We propose a novel picture of high-harmonic generation (HHG) in solids based on the concept of temporally changing band structures. To demonstrate the utility of this picture, we focus on the high-order sideband generation (HSG) caused by…
Quantum high-harmonic generation (HHG) is a prominent and growing field of research with potential capabilities of providing high photon-number entangled states of light. However, there is an open debate regarding the theory level required…
As a promising avenue to obtain new extreme ultraviolet light source and detect electronic properties, high-harmonic generation (HHG) has been actively developed in both theory and experiment. In solids lacking inversion symmetry, when…
In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of the Transformer, information originating from different tokens…
The emerging graph Transformers have achieved impressive performance for graph representation learning over graph neural networks (GNNs). In this work, we regard the self-attention mechanism, the core module of graph Transformers, as a…
High-order harmonic generation (HHG) in isolated atoms and molecules has been widely utilized in extreme ultraviolet (XUV) photonics and attosecond pulse metrology. Recently, HHG has also been observed in solids, which could lead to…
High-harmonic generation (HHG) in the two topological phases of a finite, one-dimensional, periodic structure is investigated using a self-consistent time-dependent density functional theory (TDDFT) approach. For harmonic photon energies…
Transformer architecture has emerged to be successful in a number of natural language processing tasks. However, its applications to medical vision remain largely unexplored. In this study, we present UTNet, a simple yet powerful hybrid…
We introduce Attention Graphs, a new tool for mechanistic interpretability of Graph Neural Networks (GNNs) and Graph Transformers based on the mathematical equivalence between message passing in GNNs and the self-attention mechanism in…
High-harmonic generation (HHG) is a strong-field phenomenon that is sensitive to the attosecond dynamics of tunnel ionization and coherent transport of electron-hole pairs in solids. While the foundations of solid HHG have been established,…
Multivariate time series (MTS) analysis prevails in real-world applications such as finance, climate science and healthcare. The various self-attention mechanisms, the backbone of the state-of-the-art Transformer-based models, efficiently…
We characterize the pre-softmax attention matrix $\mathbf{QK^\top}$ in transformers as an associative memory matrix encoding pairwise associations between input features. By decomposing this matrix into its symmetric and skew-symmetric…