Related papers: Re-evaluating scaling methods for distributed para…
The parallel and distributed processing are becoming de facto industry standard, and a large part of the current research is targeted on how to make computing scalable and distributed, dynamically, without allocating the resources on…
Integrating Amdahl's and Amdahl-like laws with Divisible Load Theory promises mathematical design methodology that will make possible the efficient design of tomorrow's systems.
The universal scalability law (USL) is an analytic model used to quantify application scaling. It is universal because it subsumes Amdahl's law and Gustafson linearized scaling as special cases. Using simulation, we show: (i) that the USL…
A myriad of applications ranging from engineering and scientific simulations, image and signal processing as well as high-sensitive data retrieval demand high processing power reaching up to teraflops for their efficient execution. While a…
A parallel program can be represented as a directed acyclic graph. An important performance bound is the time to execute the critical path through the graph. We show how this performance metric is related to Amdahl speedup and the degree of…
Arrival of multicore systems has enforced a new scenario in computing, the parallel and distributed algorithms are fast replacing the older sequential algorithms, with many challenges of these techniques. The distributed algorithms provide…
In the case of multi-threading as found in contemporary programming languages, parallel processes are interleaved according to what is known as a process-scheduling policy in the field of operating systems. In a previous paper, we extend…
The accelerating technological landscape and drive towards net-zero emission made the power system grow in scale and complexity. Serial computational approaches for grid planning and operation struggle to execute necessary calculations…
It is commonly believed that scaling language models should commit a significant space or time cost, by increasing the parameters (parameter scaling) or output tokens (inference-time scaling). We introduce the third and more…
The end of Moore's Law and Dennard Scaling has slowed processor improvements in the past decade. While multi-core processors have improved performance, they are limited by the application's level of parallelism, as prescribed by Amdahl's…
Recent advances in computing architectures and networking are bringing parallel computing systems to the masses so increasing the number of potential users of these kinds of systems. In particular, two important technological evolutions are…
In high performance computing environments, we observe an ongoing increase in the available numbers of cores. This development calls for re-emphasizing performance (scalability) analysis and speedup laws as suggested in the literature…
Pre-exascale High Performance Computers (HPC) can reach more than 400 Pflop/s real perfor-mance according the HPLinpack benchmarks. For nanoscience and quantum biology there are requirements for those program codes based on quantum physics…
We consider a large-scale parallel-server system, where each server independently adjusts its processing speed in a decentralized manner. The objective is to minimize the overall cost, which comprises the average cost of maintaining the…
A parallel computer system is a collection of processing elements that communicate and cooperate to solve large computational problems efficiently. To achieve this, at first the large computational problem is partitioned into several tasks…
Writing data in parallel is a common operation in some computing environments and a good proxy for a number of other parallel processing patterns. The duration of time taken to write data in large-scale compute environments can vary…
Nowadays, we are to find out solutions to huge computing problems very rapidly. It brings the idea of parallel computing in which several machines or processors work cooperatively for computational tasks. In the past decades, there are a…
In the last two decades, the continuous increase of computational power has produced an overwhelming flow of data which has called for a paradigm shift in the computing architecture and large scale data processing mechanisms. MapReduce is a…
System performance for networks composed of interconnected subsystems can be increased if the traditionally separated subsystems are jointly optimized. Recently, parallel and distributed optimization methods have emerged as a powerful tool…
While Learned Data Compression (LDC) has achieved superior compression ratios, balancing precise probability modeling with system efficiency remains challenging. Crucially, uniform single-stream architectures struggle to simultaneously…