English
Related papers

Related papers: Scheduling Coflows with Dependency Graph

200 papers

This paper introduces a parallel scheduling problem where a directed acyclic graph modeling $t$ tasks and their dependencies needs to be executed on $n$ unreliable workers. Worker $i$ executes task $j$ correctly with probability $p_{i,j}$.…

Data Structures and Algorithms · Computer Science 2007-05-23 Grzegorz Malewicz

In the past decade, increasingly network scheduling techniques have been proposed to boost the distributed application performance. Flow-level metrics, such as flow completion time (FCT), are based on the abstraction of flows yet they…

Networking and Internet Architecture · Computer Science 2019-01-18 Jiawei Fei , Yang Shi , Qun Huang , Mei Wen

Datacenter networks routinely support the data transfers of distributed computing frameworks in the form of coflows, i.e., sets of concurrent flows related to a common task. The vast majority of the literature has focused on the problem of…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-05-04 Quang-Trung Luu , Olivier Brun , Rachid El-Azouzi , Francesco De Pellegrini , Balakrishna J. Prabhu , Cédric Richier

With the rapid advancement of technology, parallel computing applications have become increasingly popular and are commonly executed in large data centers. These applications involve two phases: computation and communication, which are…

Data Structures and Algorithms · Computer Science 2023-06-16 Chi-Yeh Chen , Jun Chen

Co-flows model a modern scheduling setting that is commonly found in a variety of applications in distributed and cloud computing. In co-flow scheduling, there are $m$ input ports and $m$ output ports. Each co-flow $j \in J$ can be…

Data Structures and Algorithms · Computer Science 2018-12-04 Sungjin Im , Manish Purohit

Co-flows model a modern scheduling setting that is commonly found in a variety of applications in distributed and cloud computing. A stochastic co-flow task contains a set of parallel flows with randomly distributed sizes. Further, many…

Distributed, Parallel, and Cluster Computing · Computer Science 2018-02-26 Ruijiu Mao , Vaneet Aggarwal , Mung Chiang

To improve the application-level communication performance, scheduling of coflows, a collection of parallel flows sharing the same objective, is prevalent in modern data center networks (DCNs). Meanwhile, a hybrid-switched DCN design…

Networking and Internet Architecture · Computer Science 2023-06-19 Xin Wang , Hong Shen , Hui Tian

Coflow has emerged as a fundamental application-layer abstraction in distributed systems, representing communication dependencies and enabling collaborative management of related flows to enhance job completion efficiency. To meet the…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-04-30 Xin Wang , Hong Shen , Hui Tian , Ye Tao

In data-parallel computing frameworks, intermediate parallel data is often produced at various stages which needs to be transferred among servers in the datacenter network (e.g. the shuffle phase in MapReduce). A stage often cannot start or…

Data Structures and Algorithms · Computer Science 2017-04-28 Mehrnoosh Shafiee , Javad Ghaderi

A coflow is a collection of parallel flows belonging to the same job. It has the all-or-nothing property: a coflow is not complete until the completion of all its constituent flows. In this paper, we focus on optimizing \emph{coflow-level…

Networking and Internet Architecture · Computer Science 2017-01-11 Qingkai Liang , Eytan Modiano

We consider global fixed-priority (G-FP) scheduling of parallel tasks, in which each task is represented as a directed acyclic graph (DAG). We summarize and highlight limitations of the state-of-the-art analyses for G-FP and propose a novel…

Distributed, Parallel, and Cluster Computing · Computer Science 2019-05-14 Son Dinh , Christopher Gill , Kunal Agrawal

The scheduling and schedulability analysis of real-time directed acyclic graph (DAG) task systems have received much recent attention. The DAG model can accurately represent intra-task parallelim and precedence constraints existing in many…

Operating Systems · Computer Science 2018-08-02 Zheng Dong , Cong Liu

Motivated by applications in grid computing and project management, we study multiprocessor scheduling in scenarios where there is uncertainty in the successful execution of jobs when assigned to processors. We consider the problem of…

Distributed, Parallel, and Cluster Computing · Computer Science 2007-05-23 Guolong Lin , Rajmohan Rajaraman

Over the years, many multiprocessor locking protocols have been designed and analyzed. However, the performance of these protocols highly depends on how the tasks are partitioned and prioritized and how the resources are shared locally and…

Operating Systems · Computer Science 2018-09-11 Jian-Jia Chen , Georg von der Brüggen , Junjie Shi , Niklas Uete

Multiprocessor scheduling of hard real-time tasks modeled by directed acyclic graphs (DAGs) exploits the inherent parallelism presented by the model. For DAG tasks, a node represents a request to execute an object on one of the available…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-03-02 Corey Tessler , Venkata P. Modekurthy , Nathan Fisher , Abusayeed Saifullah

Scheduling job flows efficiently and rapidly on distributed computing clusters is one of huge challenges for daily operation of data centers. In a practical scenario, a single job consists of numerous stages with complex dependency relation…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-02-22 Jinhong Luo , Yunfan Zhou , Xijun Li , Mingxuan Yuan , Jianguo Yao , Jia Zeng

Scheduling a set of jobs over a collection of machines is a fundamental problem that needs to be solved millions of times a day in various computing platforms: in operating systems, in large data clusters, and in data centers. Along with…

Data Structures and Algorithms · Computer Science 2018-07-10 Janardhan Kulkarni , Shi Li

Distributed applications, such as database queries and distributed training, consist of both compute and network tasks. DAG-based abstraction primarily targets compute tasks and has no explicit network-level scheduling. In contrast, Coflow…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-07-16 Weitao Wang , Sushovan Das , Xinyu Crystal Wu , Zhuang Wang , Ang Chen , T. S. Eugene Ng

Scheduling computational tasks represented by directed acyclic graphs (DAGs) is challenging because of its complexity. Conventional scheduling algorithms rely heavily on simple heuristics such as shortest job first (SJF) and critical path…

Machine Learning · Computer Science 2021-03-08 Zhigang Hua , Feng Qi , Gan Liu , Shuang Yang

In latency-sensitive applications, efficient task scheduling is crucial for maintaining Quality of Service (QoS) while meeting strict timing constraints. This paper addresses the challenge of scheduling periodic tasks structured as directed…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-06-17 Ashutosh Shankar , Astha Kumari