English
Related papers

Related papers: How Workflow Engines Should Talk to Resource Manag…

200 papers

Workflow management systems (WMS) support the composition and deployment of workflow-oriented applications in distributed computing environments. They hide the complexity of managing large-scale applications, which includes the controlling…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-06-04 Muhammad H. Hilman , Maria A. Rodriguez , Rajkumar Buyya

We consider the following shared-resource scheduling problem: Given a set of jobs $J$, for each $j\in J$ we must schedule a job-specific processing volume of $v_j>0$. A total resource of $1$ is available at any time. Jobs have a resource…

Data Structures and Algorithms · Computer Science 2023-10-11 Christoph Damerius , Peter Kling , Florian Schneider

Scientific workflow management systems enable the reproducible execution of data analysis pipelines on cluster infrastructures managed by resource managers such as Kubernetes, Slurm, or HTCondor. These resource managers require resource…

Distributed, Parallel, and Cluster Computing · Computer Science 2024-10-03 Jonathan Bader , Ansgar Lößer , Lauritz Thamsen , Björn Scheuermann , Odej Kao

Due to the limited resource capacity of edge servers and the high purchase costs of edge resources, service providers are facing the new challenge of how to take full advantage of the constrained edge resources for Internet of Things (IoT)…

Distributed, Parallel, and Cluster Computing · Computer Science 2024-06-03 Lujie Tang , Minxian Xu , Chengzhong Xu , Kejiang Ye

Adaptive workloads can change on--the--fly the configuration of their jobs, in terms of number of processes. In order to carry out these job reconfigurations, we have designed a methodology which enables a job to communicate with the…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-06-01 Sergio Iserte , Rafael Mayo , Enrique S. Quintana-Orti , Vicenc Beltran , Antonio J. Peña

With the rapid advancement of technology, parallel computing applications have become increasingly popular and are commonly executed in large data centers. These applications involve two phases: computation and communication, which are…

Data Structures and Algorithms · Computer Science 2023-06-16 Chi-Yeh Chen , Jun Chen

The utilization of cloud environments to deploy scientific workflow applications is an emerging trend in scientific community. In this area, the main issue is the scheduling of workflows, which is known as an NP-complete problem. Apart from…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-01-17 J. E. Ndamlabin Mboula , V. C. Kamla , M. H. Hilman , C. Tayou Djamegni

Advance reservation is important to guarantee the quality of services of jobs by allowing exclusive access to resources over a defined time interval on resources. It is a challenge for the scheduler to organize available resources…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-03-06 Bo Li , Yijian Pei , Bin Shen , Hao Wu , Min He , Jundong Yang

In the field of computational science and engineering, workflows often entail the application of various software, for instance, for simulation or pre- and postprocessing. Typically, these components have to be combined in arbitrarily…

Software Engineering · Computer Science 2022-11-15 Philipp Diercks , Dennis Gläser , Ontje Lünsdorf , Michael Selzer , Bernd Flemisch , Jörg F. Unger

The scheduling literature has traditionally focused on a single type of resource (e.g., computing nodes). However, scientific applications in modern High-Performance Computing (HPC) systems process large amounts of data, hence have diverse…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-06-15 Lucas Perotin , Hongyang Sun , Padma Raghavan

Scheduling query execution plans is a particularly complex problem in shared-nothing parallel systems, where each site consists of a collection of local time-shared (e.g., CPU(s) or disk(s)) and space-shared (e.g., memory) resources and…

Databases · Computer Science 2014-04-01 Minos Garofalakis , Yannis Ioannidis

When multiple processor cores (CPUs) and a GPU integrated together on the same chip share the off-chip DRAM, requests from the GPU can heavily interfere with requests from the CPUs, leading to low system performance and starvation of cores.…

Hardware Architecture · Computer Science 2018-05-01 Rachata Ausavarungnirun , Gabriel H. Loh , Lavanya Subramanian , Kevin Chang , Onur Mutlu

We consider a natural scheduling problem which arises in many distributed computing frameworks. Jobs with diverse resource requirements (e.g. memory requirements) arrive over time and must be served by a cluster of servers, each with a…

Networking and Internet Architecture · Computer Science 2019-01-21 Konstantinos Psychas , Javad Ghaderi

Large Language Models (LLMs) such as GPT-4 and Llama have shown remarkable capabilities in a variety of software engineering tasks. Despite the advancements, their practical deployment faces challenges, including high financial costs, long…

Software Engineering · Computer Science 2025-08-06 Yueyue Liu , Hongyu Zhang , Yuantian Miao

In this work we study the problem of scheduling tasks with dependencies in multiprocessor architectures where processors have different speeds. We present the preemptive algorithm "Save-Energy" that given a schedule of tasks it post…

Distributed, Parallel, and Cluster Computing · Computer Science 2008-06-09 Ioannis Chatzigiannakis , Georgios Giannoulis , Paul G. Spirakis

Scientific workflow has become essential in software engineering because it provides a structured approach to designing, executing, and analyzing scientific experiments. Software developers and researchers have developed hundreds of…

Software Engineering · Computer Science 2023-09-15 Khairul Alam , Banani Roy , Alexander Serebrenik

Resource selection and task placement for distributed execution poses conceptual and implementation difficulties. Although resource selection and task placement are at the core of many tools and workflow systems, the methods are ad hoc…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-11-04 Matteo Turilli , Yadu Nand Babuji , Andre Merzky , Ming Tai Ha , Michael Wilde , Daniel S. Katz , Shantenu Jha

In High Performance Computing (HPC) infrastructures, the control of resources by batch systems can lead to prolonged queue waiting times and adverse effects on the overall execution times of applications, particularly in data-intensive and…

Distributed, Parallel, and Cluster Computing · Computer Science 2024-01-19 Abel Souza , Kristiaan Pelckmans , Devarshi Ghoshal , Lavanya Ramakrishnan , Johan Tordsson

This paper presents a multiagent approach as a paradigm for scheduling parallel jobs in a parallel system. Scheduling parallel jobs is performed as a means to balance the load of a system in order to improve the performance of a parallel…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-06-29 Jaderick P. Pabico

Scientific workflows have become essential for orchestrating complex computational processes across distributed resources, managing large datasets, and ensuring reproducibility in modern research. The Workflows Community Summit 2025, held…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-02-06 Irene Bonati , Silvina Caino-Lores , Tainã Coleman , Sagar Dolas , Sandro Fiore , Venkatesh Kannan , Marco Verdicchio , Sean R. Wilkinson , Rafael Ferreira da Silva