Related papers: Transformer-Based Neural Networks Backflow for Str…
Transformer architectures are typically described in algorithmic and statistical terms, leaving their internal mechanics without a familiar structural language for researchers trained in physical theories. To bridge this gap, we develop a…
We establish connections between the Transformer architecture, originally introduced for natural language processing, and Graph Neural Networks (GNNs) for representation learning on graphs. We show how Transformers can be viewed as message…
The projection-based wave function (WF)-in-DFT embedding enables an efficient description of both the energetics and properties of large and complex chemical systems, with accuracy exceeding that of pure DFT. Recently, we have proposed…
We investigate the effects of electronic correlations on the Bernevig-Hughes-Zhang model using the real-space density matrix renormalization group (DMRG) algorithm. We introduce a method to probe topological phase transitions in systems…
Transformers have exhibited exceptional capabilities in sequence modeling tasks, leveraging self-attention and in-context learning. Critical to this success are induction heads, attention circuits that enable copying tokens based on their…
Accurate numerical solutions for the Schr\"odinger equation are of utmost importance in quantum chemistry. However, the computational cost of current high-accuracy methods scales poorly with the number of interacting particles. Combining…
The molecular Schr\"odinger equation is rewritten in terms of non-unitary equations of motion for the nuclei (or electrons) that depend parametrically on the configuration of an ensemble of generally defined electronic (or nuclear)…
Machine-learning models in chemistry - when based on descriptors of atoms embedded within molecules - face essential challenges in transferring the quality of predictions of local electronic structures and their associated properties across…
Electronic topological phases of matter, characterized by robust boundary states derived from topologically nontrivial bulk states, are pivotal for next-generation electronic devices. However, understanding their complex quantum phases,…
The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide a novel perspective towards understanding the architecture:…
We introduce a novel computational approach for the investigation of complex correlated electron materials which makes it possible to evaluate interatomic forces and thereby determine atomic displacements and structural transformations…
The extraction of text in high quality is essential for text-based document analysis tasks like Document Classification or Named Entity Recognition. Unfortunately, this is not always ensured, as poor scan quality and the resulting artifacts…
This paper introduces a deep learning-based super-resolution (SR) framework specifically developed for accurately reconstructing high-resolution velocity fields in two-way coupled particle-laden turbulent flows. Leveraging conditional…
Fast and accurate simulation of complex chemical systems in environments such as solutions is a long standing challenge in theoretical chemistry. In recent years, machine learning has extended the boundaries of quantum chemistry by…
The electronic structure is found to be understandable in terms of free-atom term values and universal interorbital coupling parameters, since self-consistent tight-binding calculations indicate that Coulomb shifts of the d-state energies…
We present the results of numerical studies for selected materials with strongly correlated electrons using a combination of the local-density approximation and dynamical mean-field theory (DMFT). For the solution of the DMFT equations a…
State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion. Vaswani et al. (2017) propose a new architecture that avoids recurrence and convolution…
Due to its favorable computational efficiency time-dependent (TD) density functional theory (DFT) enables the prediction of electronic spectra in a high-throughput manner across chemical space. Its predictions, however, can be quite…
The inverse design of metamaterial architectures presents a significant challenge, particularly for nonlinear mechanical properties involving large deformations, buckling, contact, and plasticity. Traditional methods, such as gradient-based…
Recent advancements in attention mechanisms have replaced recurrent neural networks and its variants for machine translation tasks. Transformer using attention mechanism solely achieved state-of-the-art results in sequence modeling. Neural…