Related papers: Transformer-based prediction of two-dimensional ma…

Learning feature interaction is the critical backbone to building recommender systems. In web-scale applications, learning feature interaction is extremely challenging due to the sparse and large input feature space; meanwhile, manually…

Quantum mechanical methods based on the density functional theory (DFT) offer a realistic possibility of first-principles design of organic donor-acceptor systems and engineered band-gap materials. This promise is contingent upon the…

Linearization of attention using various kernel approximation and kernel learning techniques has shown promise. Past methods used a subset of combinations of component functions and weight matrices within the random feature paradigm. We…

This paper investigates automatic piano transcription based on computationally-efficient yet high-performant variants of the Transformer that can capture longer-term dependency over the whole musical piece. Recently, transformer-based…

Two-dimensional (2D) materials have wide applications in superconductors, quantum, and topological materials. However, their rational design is not well established, and currently less than 6,000 experimentally synthesized 2D materials have…

Heterogeneous interfaces are central to many energy-related applications in the nanoscale. From the first-principles electronic structure perspective, one of the outstanding problems is accurately and efficiently calculating how the…

Traditional statistical optimization-based state estimation (DSSE) algorithms rely on detailed grid parameters and mathematical assumptions of all possible uncertainties. Furthermore, random data missing due to communication failures,…

Two-Dimensional (2D) Discrete Fourier Transform (DFT) is a basic and computationally intensive algorithm, with a vast variety of applications. 2D images are, in general, non-periodic, but are assumed to be periodic while calculating their…

Transformer-based methods have demonstrated superior performance for monocular 3D object detection recently, which aims at predicting 3D attributes from a single 2D image. Most existing transformer-based methods leverage both visual and…

Vision Transformers (ViTs) have revolutionized computer vision, yet their self-attention mechanism lacks explicit spatial inductive biases, leading to suboptimal performance on spatially-structured tasks. Existing approaches introduce…

Density functional theory (DFT) is the de facto approach for predicting self-consistent-field electronic structures of ground-state configurations of complex atoms, molecules, and solids and providing their property data for materials…

At present, people usually use some methods based on convolutional neural networks (CNNs) for Electroencephalograph (EEG) decoding. However, CNNs have limitations in perceiving global dependencies, which is not adequate for common EEG…

The density functional theory (DFT) approximations that are the most accurate for the calculation of band gap of bulk materials are hybrid functionals like HSE06, the MBJ potential, and the GLLB-SC potential. More recently, generalized…

Accurate beam prediction is essential for maintaining reliable links and high spectral efficiency in dynamic low-altitude wireless networks. However, existing approaches often fail to capture the deep correlations across heterogeneous…

Multivariate time series (MTS) analysis prevails in real-world applications such as finance, climate science and healthcare. The various self-attention mechanisms, the backbone of the state-of-the-art Transformer-based models, efficiently…

High performance sheet metals with a multi-phase microstructure suffer from deformation induced damage formation during forming in the constituent phases but importantly also where these intersect. To capture damage in terms of the physical…

Since their introduction the Trasformer architectures emerged as the dominating architectures for both natural language processing and, more recently, computer vision applications. An intrinsic limitation of this family of "fully-attentive"…

Deformation component analysis is a fundamental problem in geometry processing and shape understanding. Existing approaches mainly extract deformation components in local regions at a similar scale while deformations of real-world objects…

Transformer-based models have been widely adopted for sentiment analysis tasks due to their exceptional ability to capture contextual information. However, these methods often exhibit suboptimal accuracy in certain scenarios. By analyzing…

Deep learning models for channel estimation in Orthogonal Frequency Division Multiplexing (OFDM) systems often suffer from performance degradation under fast-fading channels and low-SNR scenarios. To address these limitations, we introduce…