Related papers: The Ghost of Performance Reproducibility Past
Performance variability has been acknowledged as a problem for over a decade by cloud practitioners and performance engineers. Yet, our survey of top systems conferences reveals that the research community regularly disregards variability…
Stochastic fluctuations of molecular abundances are a ubiquitous feature of cellular processes and lead to significant cell-to-cell variability. Recent theoretical work established lower bounds for stochastic fluctuations in cells for broad…
There are many science applications that require scalable task-level parallelism and support for flexible execution and coupling of ensembles of simulations. Most high-performance system software and middleware, however, are designed to…
Using an extremely large number of processing elements in computing systems leads to unexpected phenomena, such as different efficiencies of the same system for different tasks, that cannot be explained in the frame of classical computing…
Performance benchmarking is a common practice in software engineering, particularly when building large-scale, distributed, and data-intensive systems. While cloud environments offer several advantages for running benchmarks, it is often…
In the recent years it can be observed increasing popularity of parallel processing using multi-core processors, local clusters, GPU and others. Moreover, currently one of the main requirements the IT users is the reduction of maintaining…
Computational models in chemistry rely on a number of approximations. The effect of such approximations on observables derived from them is often unpredictable. Therefore, it is challenging to quantify the uncertainty of a computational…
When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest validation score. However, downstream metrics often exhibit…
Fractal properties are usually characterized by means of various statistical tools which deal with spatial average quantities. Here we focus on the determination of fluctuations around the average counts and we develop a test for the study…
The dynamics of adaptation is difficult to predict because it is highly stochastic even in large populations. The uncertainty emerges from number fluctuations, called genetic drift, arising in the small number of particularly fit…
What is called "numerical reproducibility" is the problem of getting the same result when the scientific computation is run several times, either on the same machine or on different machines, with different types and numbers of processing…
We present a dynamical theory of complex cooperative projects such as large engineering design or software development efforts, comprised of concurrent and interrelated tasks. The model accounts for temporal fluctuations both in task…
We investigate the model of "reversible ratchet" with interacting particles, introduced by us earlier [Europhys. Lett. 84, 50009 (2008)]. We further clarify the effect of efficiency enhancement due to interaction and show that it is of…
The volume fluctuations in statistical mechanics are discussed. First, the volume fluctuations in ensembles with a fixed external pressure, the so called pressure ensembles, are considered. Second, a generalization of the pressure ensembles…
Research processes often rely on high-performance computing (HPC), but HPC is often seen as antithetical to "reproducibility": one would have to choose between software that achieves high performance, and software that can be deployed in a…
Context: Performance regressions negatively impact execution time and memory usage of software systems. Nevertheless, there is a lack of systematic methods to evaluate the effectiveness of performance test suites. Performance mutation…
We discuss some of the fundamental practical limitations of the Fluctuation-Enhanced Sensing of odors and gases. We address resolution, measurement speed, reproducibility, memory and other problems such as humidity. Various techniques and…
We present a brief introduction to the statistical properties of systems with large fluctuations. We point out that for such systems the relevant statistical quantities are scaling exponents and the nature of fluctuations is completely…
Important computational physics problems are often large-scale in nature, and it is highly desirable to have robust and high performing computational frameworks that can quickly address these problems. However, it is no trivial task to…
The performance of spectral clustering relies on the fluctuations of the entries of the eigenvectors of a similarity matrix, which has been left uncharacterized until now. In this letter, it is shown that the signal $+$ noise structure of a…