Related papers: The Ghost of Performance Reproducibility Past
Software for the resolution of certain kind of problems, those that rate high in the Stringent Performance Objectives adjustment factor (IFPUG scheme), can be described using a combination of game theory and autonomous systems. From this…
We propose a rigorous decomposition of predictive error, highlighting that not all 'irreducible' error is genuinely immutable. Many domains stand to benefit from iterative enhancements in measurement, construct validity, and modeling. Our…
We review here the recent success in quantum annealing, i.e., optimization of the cost or energy functions of complex systems utilizing quantum fluctuations. The concept is introduced in successive steps through the studies of mapping of…
Given the continuous increase in dataset sizes and the complexity of forecasting models, the trade-off between forecast accuracy and computational cost is emerging as an extremely relevant topic, especially in the context of ensemble…
Standard results for the fluctuations of thermodynamic quantities are derived under the assumption of sampling identical systems that are in different, not fully equilibrated states. These results apply to fluctuations with time in a…
Reproducibility in research remains hindered by complex systems involving data, models, tools, and algorithms. Studies highlight a reproducibility crisis due to a lack of standardized reporting, code and data sharing, and rigorous…
Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are both accurate and reproducible. We demonstrate that the…
Modern software systems are often highly configurable to tailor varied requirements from diverse stakeholders. Understanding the mapping between configurations and the desired performance attributes plays a fundamental role in advancing the…
Due to the diffusion of IoT, modern software systems are often thought to control and coordinate smart devices in order to manage assets and resources, and to guarantee efficient behaviours. For this class of systems, which interact…
A driven granular material, e.g. a vibrated box full of sand, is a stationary system which may be very far from equilibrium. The standard equilibrium statistical mechanics is therefore inadequate to describe fluctuations in such a system.…
The adoption of heterogeneous computing systems based on diverse architectures to achieve exascale computing power has worsened the performance portability problem of scientific applications that were designed to run on these platforms. To…
Scaling recommendation models into large recommendation models has become one of the most widely discussed topics. Recent efforts focus on components beyond the scaling embedding dimension, as it is believed that scaling embedding may lead…
Measuring and analyzing the performance of software has reached a high complexity, caused by more advanced processor designs and the intricate interaction between user programs, the operating system, and the processor's microarchitecture.…
Returning a system to a desired state under a force field involves a thermodynamic cost, i.e., {\it work}. This cost fluctuates for a small-scale system from one experimental realization to another. We introduce a general framework to…
At face value, this essay is about understanding a fairly esoteric governance tool called compute thresholds. However, in order to grapple with whether these thresholds will achieve anything, we must first understand how they came to be. To…
The global energy fluctuations of a low density gas granular gas in the homogeneous cooling state near its clustering instability are studied by means of molecular dynamics simulations. The relative dispersion of the fluctuations is shown…
We show that differently constructed ensembles having the same density matrix may be physically distinguished by observing fluctuations of some observables. An explicit expression for fluctuations of an observable in an ensemble is given.…
Understanding how software defects manifest and evolve in production environments is critical for improving reliability. While previous research has largely focused on pre-release defects, the nature of residual faults, i.e., those escaping…
From the perspective of quantum thermodynamics, realisable measurements cost work and result in measurement devices that are not perfectly correlated with the measured systems. We investigate the consequences for the estimation of work in…
Optimization of complex functions, such as the output of computer simulators, is a difficult task that has received much attention in the literature. A less studied problem is that of optimization under unknown constraints, i.e., when the…