Related papers: Self-attention enabled quantum path analysis of hi…
The Higgs self-coupling is crucial for understanding the structure of the scalar potential and the mechanism of electroweak symmetry breaking. In this work, utilizing deep neural network based on Particle Transformer that relies on…
Transformers have demonstrated remarkable performance across diverse domains. The key component of Transformers is self-attention, which learns the relationship between any two tokens in the input sequence. Recent studies have revealed that…
Self-attention is an attention mechanism that learns a representation by relating different positions in the sequence. The transformer, which is a sequence model solely based on self-attention, and its variants achieved state-of-the-art…
Complex-valued signals encode both amplitude and phase, yet most deep models treat attention as real-valued correlation, overlooking interference effects. We introduce the Holographic Transformer, a physics-inspired architecture that…
Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…
The self-attention mechanism, a cornerstone of Transformer-based state-of-the-art deep learning architectures, is largely heuristic-driven and fundamentally challenging to interpret. Establishing a robust theoretical foundation to explain…
The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing computational overhead. Self-attention integrates learnable query and key matrices to…
High-order harmonic generation (HHG) in solids has emerged as a versatile platform for exploring ultrafast and quantum-coherent phenomena in condensed matter. Recent advances reveal Berry-phase and topological effects in harmonic emission,…
We introduce $\Sigma$-Attention, a Transformer-based operator-learning framework to address a key computational challenge in correlated materials. Our approach utilizes an Encoder-Only Transformer as an ansatz to approximate the self-energy…
Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-resolution image…
Learning reduced descriptions of chaotic many-body dynamics is fundamentally challenging: although microscopic equations are Markovian, collective observables exhibit strong memory and exponential sensitivity to initial conditions and…
Transformers are built upon multi-head scaled dot-product attention and positional encoding, which aim to learn the feature representations and token dependencies. In this work, we focus on enhancing the distinctive representation by…
We analytically and numerically investigate the emission of high-harmonic radiation from model solids by intense few-cycle mid-infrared laser pulses. In single-active-electron approximation, we expand the active electron's wavefunction in a…
State-of-the-art experiments employ strong ultrafast optical fields to study the nonlinear response of electrons in solids on an attosecond time-scale. Notably, a recent experiment retrieved a 3rd order nonlinear susceptibility by comparing…
We establish time-resolved high harmonic generation (tr-HHG) as a powerful spectroscopy for photoinduced dynamics in strongly correlated materials through a detailed investigation of the insulator-to-metal transitions in vanadium dioxide.…
Using machine learning, we explore the utility of various deep neural networks (NN) when applied to high harmonic generation (HHG) scenarios. First, we train the NNs to predict the time-dependent dipole and spectra of HHG emission from…
The paradigm of Transformers using the self-attention mechanism has manifested its advantage in learning graph-structured data. Yet, Graph Transformers are capable of modeling full range dependencies but are often deficient in extracting…
High-harmonic generation (HHG) is a coherent optical process in which the incident photon energy is up-converted to the multiples of its initial energy. In solids, under the influence of a strong laser field, electron-hole (e-h) pairs are…
High-order harmonic generation (HHG) results from strong-field laser matter interaction and it is one of the main processes that are used to extract electron structural and dynamical information about the atomic or molecular targets with…
Transformer self-attention can be interpreted as a gradient flow on the unit sphere, in which tokens evolve under softmax interaction potentials and tend to form clusters. While prior work has established clustering behavior for single-head…