Related papers: Self-attention enabled quantum path analysis of hi…
QCAAPatchTF is a quantum attention network integrated with an advanced patch-based transformer, designed for multivariate time series forecasting, classification, and anomaly detection. Leveraging quantum superpositions, entanglement, and…
Multi-scale simulations of nonlinear heterogeneous materials and composites are challenging due to the prohibitive computational costs of high-fidelity simulations. Recently, machine learning (ML) based approaches have emerged as promising…
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set,…
This paper presents the innovative HPCNeuroNet model, a pioneering fusion of Spiking Neural Networks (SNNs), Transformers, and high-performance computing tailored for particle physics, particularly in particle identification from detector…
We theoretically study high-harmonic generation (HHG) from transition metal elements Mn and Mn$^+$, using full-dimensional, all-electron, first-principles simulations. The HHG spectra calculated with the time-dependent complete-active-space…
High-harmonic generation (HHG) in solids has rapidly emerged as a promising platform for creating compact attosecond sources and probing ultrafast electron dynamics. Resonant metasurfaces are essential for enhancement of the otherwise small…
Heterogeneous graph neural networks (HeteGNNs) have demonstrated strong abilities to learn node representations by effectively extracting complex structural and semantic information in heterogeneous graphs. Most of the prevailing HeteGNNs…
High-harmonic generation (HHG) in condensed-matter systems is both a source of fundamental insight into quantum electron motion and a promising candidate to realize compact ultraviolet and ultrafast light sources. Here we argue that the…
The mechanism underlying high harmonic generation (HHG) in gases has been well clarified, characterizing attosecond pulse trains (APT) in the time domain, significantly advances the synthesis of isolated attosecond pulse (IAP). However, the…
Graph Neural Networks (GNNs) have shown remarkable success in graph representation learning. Unfortunately, current weight assignment schemes in standard GNNs, such as the calculation based on node degrees or pair-wise representations, can…
The versatility of self-attention mechanism earned transformers great success in almost all data modalities, with limitations on the quadratic complexity and difficulty of training. Efficient transformers, on the other hand, often rely on…
Deployments of artificial intelligence in medical diagnostics mandate not just accuracy and efficacy but also trust, emphasizing the need for explainability in machine decisions. The recent trend in automated medical image diagnostics leans…
We propose an extension to the transformer neural network architecture for general-purpose graph learning by adding a dedicated pathway for pairwise structural information, called edge channels. The resultant framework - which we call…
We develop a numerical scheme to investigate the high-order harmonic generation (HHG) in intense laser-matter interactions. Tracing the time evolution of every electronic laser-field-free state, we "observe" the HHG in a time-integrated…
High harmonic generation (HHG) is a powerful probe of electron dynamics on attosecond to femtosecond timescales and has been successfully used to detect electronic and structural changes in solid-state quantum materials, including…
Quantum-optical descriptions of strong-field processes have attracted significant attention in recent years. Typically, the theoretical modeling has been conducted in the Schr\"odinger picture, where results are only obtainable under…
Multi-head, key-value attention is the backbone of the widely successful Transformer model and its variants. This attention mechanism uses multiple parallel key-value attention blocks (called heads), each performing two fundamental…
Recently, Transformers have shown promising performance in various vision tasks. To reduce the quadratic computation complexity caused by each query attending to all keys/values, various methods have constrained the range of attention…
Transformers are groundbreaking architectures that have changed a flow of deep learning, and many high-performance models are developing based on transformer architectures. Transformers implemented only with attention with encoder-decoder…
While the Self-Attention mechanism in the Transformer model has proven to be effective in many domains, we observe that it is less effective in more diverse settings (e.g. multimodality) due to the varying granularity of each token and the…