English
Related papers

Related papers: Hadoop Scheduling Base On Data Locality

200 papers

Scheduling deep learning (DL) models to train on powerful clusters with accelerators like GPUs and TPUs, presently falls short, either lacking fine-grained heterogeneity awareness or leaving resources substantially under-utilized. To fill…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-03-17 Abeda Sultana , Nabin Pakka , Fei Xu , Xu Yuan , Li Chen , Nian-Feng Tzeng

Runtime scheduling and workflow systems are an increasingly popular algorithmic component in HPC because they allow full system utilization with relaxed synchronization requirements. There are so many special-purpose tools for task…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-11-03 David M. Rogers

We tackle in this paper an online network resource allocation problem with job transfers. The network is composed of many servers connected by communication links. The system operates in discrete time; at each time slot, the administrator…

Machine Learning · Statistics 2023-11-17 Ahmed Sid-Ali , Ioannis Lambadaris , Yiqiang Q. Zhao , Gennady Shaikhet , Amirhossein Asgharnia

Geo-distributed computing, a paradigm that assigns computational tasks to globally distributed nodes, has emerged as a promising approach in cloud computing, edge computing, cloud-edge computing and supercomputer computing (HPC). It enables…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-01-28 Yujian Wu , Shanjiang Tang , Ce Yu , Bin Yang , Chao Sun , Jian Xiao , Hutong Wu

Using memory located on remote machines, or far memory, as a swap space is a promising approach to meet the increasing memory demands of modern datacenter applications. Operating systems have long relied on prefetchers to mask the increased…

Many complex activities of production cycles, such as quality control or fault analysis, require highly experienced specialists to perform various operations on (semi)finished products using different tools. In practical scenarios, the…

Artificial Intelligence · Computer Science 2021-01-27 Giulia Francescutto , Konstantin Schekotihin , Mohammed M. S. El-Kholany

Computational Grids are a new trend in distributed computing systems. They allow the sharing of geographically distributed resources in an efficient way, extending the boundaries of what we perceive as distributed computing. Various…

Distributed, Parallel, and Cluster Computing · Computer Science 2014-08-24 D. Thilagavathi , Antony Selvadoss Thanamani

The ever-growing processing power of supercomputers in recent decades enables us to explore increasing complex scientific problems. Effective scheduling these jobs is crucial for individual job performance and system efficiency. The…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-09-21 Yuping Fan

In this paper we look at the problem of scheduling tasks on a single-processor system, where each task requires unit time and must be scheduled within a certain time window, and each task can be added to or removed from the system at any…

Data Structures and Algorithms · Computer Science 2016-02-16 Wei Quan Lim , Seth Gilbert , Wei Zhong Lim

Modern computing systems process jobs with resource requirements such as CPU and memory, which are described by multiresource jobs (MRJ) queueing models. In practice, job resource requirements are spread out over so many values, that it is…

Performance · Computer Science 2026-05-22 Heyuan Yao , Willow Kowalik , Izzy Grosof

Performing efficient resource provisioning is a fundamental aspect for any resource provider. Local Resource Management Systems (LRMS) have been used in data centers for decades in order to obtain the best usage of the resources, providing…

In many scheduling applications, minimizing delays is of high importance. One adverse effect of such delays is that the reward for completion of a job may decay over time. Indeed in healthcare settings, delays in access to care can result…

Systems and Control · Computer Science 2016-10-24 Neal Master , Carri W. Chan , Nicholas Bambos

The multiprocessor Fixed-Job Priority (FJP) scheduling of real-time systems is studied. An important property for the schedulability analysis, the predictability (regardless to the execution times), is studied for heterogeneous…

Operating Systems · Computer Science 2009-08-26 Liliana Cucu-Grosjean , Joël Goossens

Today high-performance computing (HPC) platforms are still dominated by batch jobs. Accordingly, effective batch job scheduling is crucial to obtain high system efficiency. Existing HPC batch job schedulers typically leverage heuristic…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-09-03 Di Zhang , Dong Dai , Youbiao He , Forrest Sheng Bao , Bing Xie

In this report we investigate the performance of Hadoop clusters, deployed with separated storage and compute layers, on top of a hypervisor managing a single physical host. We have analyzed and evaluated the different Hadoop cluster…

Distributed, Parallel, and Cluster Computing · Computer Science 2014-11-17 Todor Ivanov , Roberto V. Zicari , Sead Izberovic , Karsten Tolle

In order to fully exploit the advantages inherent to cooperating heterogeneous multi-robot teams, sophisticated coordination algorithms are essential. Time-extended multi-robot task allocation approaches assign and schedule a set of tasks…

Systems and Control · Electrical Eng. & Systems 2020-05-11 Esther Bischoff , Fabian Meyer , Jairo Inga , Sören Hohmann

In this paper, we study an optimal online resource reservation problem in a simple communication network. The network is composed of two compute nodes linked by a local communication link. The system operates in discrete time; at each time…

Optimization and Control · Mathematics 2024-04-04 Ahmed Sid-Ali , Ioannis Lambadaris , Yiqiang Q. Zhao , Gennady Shaikhet , Shima Kheradmand

We consider the problem of scheduling multiprocessor jobs to minimize the total completion time under the given energy budget. Each multiprocessor job requires more than one processor at the same moment of time. Processors may operate at…

Optimization and Control · Mathematics 2021-07-22 Alexander Kononov , Yulia Kovalenko

This paper addresses the computational offloading of Deep Neural Networks (DNNs) to nearby devices with similar processing capabilities, to avoid the larger communication delays incurred for cloud offloading. We present a preemption aware…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-04-24 Jamie Cotter , Ignacio Castineiras , Donna O'Shea , Victor Cionca

In this work we are interested in the problem of scheduling and redistributing data on master-slave platforms. We consider the case were the workers possess initial loads, some of which having to be redistributed in order to balance their…

Distributed, Parallel, and Cluster Computing · Computer Science 2016-08-16 Loris Marchal , Veronika Rehn , Yves Robert , Frédéric Vivien