Related papers: JanusPipe: Efficient Pipeline Parallel Training fo…

Deep neural networks with large model sizes achieve state-of-the-art results for tasks in computer vision (CV) and natural language processing (NLP). However, these large-scale models are too compute- or memory-intensive for…

Communication is a key bottleneck for distributed graph neural network (GNN) training. This paper proposes GNNPipe, a new approach that scales the distributed full-graph deep GNN training. Being the first to use layer-level model…

Machine-learning potentials (MLIPs) have been a breakthrough for computational physics in bringing the accuracy of quantum mechanics to atomistic modeling. To achieve near-quantum accuracy, it is necessary that neighborhoods contained in…

Titanium MXenes are two-dimensional inorganic structures composed of titanium and carbon or nitrogen elements, with distinctive electronic, thermal and mechanical properties. Despite the extensive experimental investigation, there is a…

Machine learning interatomic potentials (MLIPs) have been widely used to facilitate large-scale molecular simulations with accuracy comparable to ab initio methods. In practice, MLIP-based molecular simulations often encounter the issue of…

Machine learning interatomic potentials (MLIPs) are used to estimate potential energy surfaces (PES) from ab initio calculations, providing near quantum-level accuracy with reduced computational costs. However, the high cost of assembling…

The rapid development and large body of literature on machine learning interatomic potentials (MLIPs) can make it difficult to know how to proceed for researchers who are not experts but wish to use these tools. The spirit of this review is…

A machine-learned interatomic potential (MLIP) for multilayer MoS2 was developed using the ultra-fast force field (UF3) framework. The UF3 MLIP reproduces key properties in strong agreement with DFT including lattice constants, interlayer…

In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs…

Central to interatomic potential efficiency is the radial envelope function that enables linear scaling with computational cost by defining a local neighborhood of atoms. This has enabled MLIPs to revolutionize materials science over the…

Pipeline parallelism is an essential technique in the training of large-scale Transformer models. However, it suffers from imbalanced memory consumption, leading to insufficient memory utilization. The BPipe technique was proposed to…

Developing machine learning interatomic potentials (MLIPs) for complex materials systems remains challenging because it requires expertise in atomistic simulations, machine learning, and workflow design, as well as iterative active learning…

While machine learning (ML) interatomic potentials (IPs) are able to achieve accuracies nearing the level of noise inherent in the first-principles data to which they are trained, it remains to be shown if their increased complexities are…

Hybrid parallelism underpins large-scale LLM training across tens of thousands of GPUs. At such scale, hardware failures on individual devices lead to performance skew across devices, diminishing overall training efficiency. Existing…

As with many parts of the natural sciences, machine learning interatomic potentials (MLIPs) are revolutionizing the modeling of molecular crystals. However, challenges remain for the accurate and efficient calculation of sublimation…

Accurate prediction of surface energies and stabilities is essential for materials design, yet first-principles calculations remain computationally expensive and most existing interatomic potentials are trained only on bulk systems. Here,…

Frontier models increasingly adopt Mixture-of-Experts (MoE) architectures to achieve large-model performance at reduced cost. However, training MoE models on HPC platforms is hindered by large memory footprints, frequent large-scale…

Mixed-Integer Linear Programming (MILP) is a foundational tool for complex decision-making problems. However, the NP-hard nature of MILP presents a significant computational challenge, motivating the development of machine learning-based…

Memory-based Temporal Graph Neural Networks (MTGNNs) are a class of temporal graph neural networks that utilize a node memory module to capture and retain long-term temporal dependencies, leading to superior performance compared to…

The core of molecular dynamics simulation fundamentally lies in the interatomic potential. Traditional empirical potentials lack accuracy, while first-principles methods are computationally prohibitive. Machine learning interatomic…