Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development
Abstract
Scientific workflows are a cornerstone of modern scientific computing, and they have underpinned some of the most significant discoveries of the last decade. Many of these workflows have high computational, storage, and/or communication demands, and thus must execute on a wide range of large-scale platforms, from large clouds to upcoming exascale HPC platforms. Workflows will play a crucial role in the data-oriented and post-Moore's computing landscape as they democratize the application of cutting-edge research techniques, computationally intensive methods, and use of new computing platforms. As workflows continue to be adopted by scientific projects and user communities, they are becoming more complex. Workflows are increasingly composed of tasks that perform computations such as short machine learning inference, multi-node simulations, long-running machine learning model training, amongst others, and thus increasingly rely on heterogeneous architectures that include CPUs but also GPUs and accelerators. The workflow management system (WMS) technology landscape is currently segmented and presents significant barriers to entry due to the hundreds of seemingly comparable, yet incompatible, systems that exist. Another fundamental problem is that there are conflicting theoretical bases and abstractions for a WMS. Systems that use the same underlying abstractions can likely be translated between, which is not the case for systems that use different abstractions. More information: https://workflowsri.org/summits/technical
Keywords
Cite
@article{arxiv.2106.05177,
title = {Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development},
author = {Rafael Ferreira da Silva and Henri Casanova and Kyle Chard and Tainã Coleman and Dan Laney and Dong Ahn and Shantenu Jha and Dorran Howell and Stian Soiland-Reys and Ilkay Altintas and Douglas Thain and Rosa Filgueira and Yadu Babuji and Rosa M. Badia and Bartosz Balis and Silvina Caino-Lores and Scott Callaghan and Frederik Coppens and Michael R. Crusoe and Kaushik De and Frank Di Natale and Tu M. A. Do and Bjoern Enders and Thomas Fahringer and Anne Fouilloux and Grigori Fursin and Alban Gaignard and Alex Ganose and Daniel Garijo and Sandra Gesing and Carole Goble and Adil Hasan and Sebastiaan Huber and Daniel S. Katz and Ulf Leser and Douglas Lowe and Bertram Ludaescher and Ketan Maheshwari and Maciej Malawski and Rajiv Mayani and Kshitij Mehta and Andre Merzky and Todd Munson and Jonathan Ozik and Loïc Pottier and Sashko Ristov and Mehdi Roozmeh and Renan Souza and Frédéric Suter and Benjamin Tovar and Matteo Turilli and Karan Vahi and Alvaro Vidal-Torreira and Wendy Whitcup and Michael Wilde and Alan Williams and Matthew Wolf and Justin Wozniak},
journal= {arXiv preprint arXiv:2106.05177},
year = {2021}
}