English

AI-coupled HPC Workflows

Distributed, Parallel, and Cluster Computing 2022-08-26 v1 Artificial Intelligence Machine Learning Software Engineering

Abstract

Increasingly, scientific discovery requires sophisticated and scalable workflows. Workflows have become the ``new applications,'' wherein multi-scale computing campaigns comprise multiple and heterogeneous executable tasks. In particular, the introduction of AI/ML models into the traditional HPC workflows has been an enabler of highly accurate modeling, typically reducing computational needs compared to traditional methods. This chapter discusses various modes of integrating AI/ML models to HPC computations, resulting in diverse types of AI-coupled HPC workflows. The increasing need of coupling AI/ML and HPC across scientific domains is motivated, and then exemplified by a number of production-grade use cases for each mode. We additionally discuss the primary challenges of extreme-scale AI-coupled HPC campaigns -- task heterogeneity, adaptivity, performance -- and several framework and middleware solutions which aim to address them. While both HPC workflow and AI/ML computing paradigms are independently effective, we highlight how their integration, and ultimate convergence, is leading to significant improvements in scientific performance across a range of domains, ultimately resulting in scientific explorations otherwise unattainable.

Keywords

Cite

@article{arxiv.2208.11745,
  title  = {AI-coupled HPC Workflows},
  author = {Shantenu Jha and Vincent R. Pascuzzi and Matteo Turilli},
  journal= {arXiv preprint arXiv:2208.11745},
  year   = {2022}
}
R2 v1 2026-06-25T01:57:11.213Z