Related papers: Transformer-based prediction of two-dimensional ma…

The stochastic density functional theory (DFT) [Phys. Rev. Lett. 111, 106402 (2013)] is a valuable linear scaling approach to Kohn-Sham DFT that does not rely on the sparsity of the density matrix. Linear (and often sub-linear) scaling is…

Transformer-based models have become the state of the art across multiple domains, from natural language processing to machine listening, thanks to the attention mechanisms. However, the attention layers require a large number of parameters…

Long-term time series forecasting (LTSF) is a crucial aspect of modern society, playing a pivotal role in facilitating long-term planning and developing early warning systems. While many Transformer-based models have recently been…

One of the enticing features common to most of the two-dimensional electronic systems that are currently at the forefront of materials science research is the ability to easily introduce a combination of planar deformations and bending in…

While self-attention has been instrumental in the success of Transformers, it can lead to over-concentration on a few tokens during training, resulting in suboptimal information flow. Enforcing doubly-stochastic constraints in attention…

Atomic modulations of two-dimensional materials using scanning tunneling microscope (STM) tip-induced forces modifies their mechanical and electrical properties. In situ topographic and spectroscopic probing through electrical tunneling has…

Differential Transformer has recently gained significant attention for its impressive empirical performance, often attributed to its ability to perform noise canceled attention. However, precisely how differential attention achieves its…

Accurate segmentation is a crucial step in medical image analysis and applying supervised machine learning to segment the organs or lesions has been substantiated effective. However, it is costly to perform data annotation that provides…

Learning solution operators for systems with complex, varying geometries and parametric physical settings is a central challenge in scientific machine learning. In many-query regimes such as design optimization, control and inverse…

Recently Transformers have provided state-of-the-art performance in sparse matching, crucial to realize high-performance 3D vision applications. Yet, these Transformers lack efficiency due to the quadratic computational complexity of their…

Herein, we demonstrate that first-principles calculations can be used for mapping electronic properties of two-dimensional (2d) materials with respect to non-uniform strain. By investigating four representative single-layer 2d compounds…

The quadratic computational cost of the self-attention mechanism is a primary challenge in scaling Transformer models. While attention sparsity is widely studied as a technique to improve computational efficiency, it is almost universally…

Electronic density of states (DOS) is a key factor in condensed matter physics and material science that determines the properties of metals. First-principles density-functional theory (DFT) calculations have typically been used to obtain…

Designing model architectures requires decisions such as selecting operators (e.g., attention, convolution) and configurations (e.g., depth, width). However, evaluating the impact of these decisions on model quality requires costly…

Using density functional theory (DFT), we investigate mechanical properties of a few 2D metal-organic frameworks (MOFs) and covalent-organic frameworks (COFs) having Dirac and flat bands. These porous materials have become a subject of…

Detection Transformers represent end-to-end object detection approaches based on a Transformer encoder-decoder architecture, exploiting the attention mechanism for global relation modeling. Although Detection Transformers deliver results on…

Applying elastic deformation can tune a material physical properties locally and reversibly. Spatially modulated lattice deformation can create a bandgap gradient, favouring photo-generated charge separation and collection in optoelectronic…

Accurate predictions of material properties within the chemical space of transition metal fluorides (TMFs), using density functional theory (DFT) is important for advancing several technological applications. The state-of-the-art semi-local…

Accurate traffic flow prediction is essential for applications like transport logistics but remains challenging due to complex spatio-temporal correlations and non-linear traffic patterns. Existing methods often model spatial and temporal…

Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose theory-guided deep-learning load…