Related papers: The Ghost of Performance Reproducibility Past
Cloud benchmarks suffer from performance fluctuations caused by resource contention, network latency, hardware heterogeneity, and other factors along with decisions taken in the benchmark design. In particular, the sampling strategy of…
Benchmarks and performance experiments are frequently conducted in cloud environments. However, their results are often treated with caution, as the presumed high variability of performance in the cloud raises concerns about reproducibility…
The increasing complexity of the software/hardware stack of modern supercomputers results in explosion of parameters. The performance analysis becomes a truly experimental science, even more challenging in the presence of massive…
The fluctuation theorems have remained one of the cornerstones in the study of systems that are driven far out of equilibrium, and they provide strong constraints on the fraction of trajectories that behave atypically in light of the second…
We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone operations such as inexact or stochastic gradient computations…
Computer-use agents have rapidly improved on real-world tasks such as web navigation, desktop automation, and software interaction, in some cases surpassing human performance. Yet even when the task and model are unchanged, an agent that…
Modern computer systems are highly configurable, with the total variability space sometimes larger than the number of atoms in the universe. Understanding and reasoning about the performance behavior of highly configurable systems, over a…
Nanoscale machines are strongly influenced by thermal fluctuations, contrary to their macroscopic counterparts. As a consequence, even the efficiency of such microscopic machines becomes a fluctuating random variable. Using geometric…
Generative AI systems are transforming content creation, but their usability remains a key challenge. This paper examines usability factors such as user experience, transparency, control, and cognitive load. Common challenges include…
Quantum computation has made considerable progress in the last decade with multiple emerging technologies providing proof-of-principle experimental demonstrations of such calculations. However, these experimental demonstrations of quantum…
It is well recognized that the project productivity is a key driver in estimating software project effort from Use Case Point size metric at early software development stages. Although, there are few proposed models for predicting…
As applications grow in capability, they also grow in complexity. This complexity in turn gets pushed into modules and libraries. In addition, hardware configurations become increasingly elaborate, too. These two trends make understanding,…
We suggest an extension of the standard concept of statistical ensembles. Namely, we introduce a class of ensembles with extensive quantities fluctuating according to an externally given distribution. As an example the influence of energy…
Using the fluctuation theorem supplemented with geometric arguments, we derive universal features of the (long-time) efficiency fluctuations for thermal and isothermal machines operating under steady or periodic driving, close or far from…
One of the difficulties in developing collective adaptive systems is the challenge of simultaneously engineering both the desired resilient behaviour of the collective and the details of its implementation on individual devices. Aggregate…
Voltage fluctuations are one of the most common low-frequency disturbances of power quality. Diagnostics of these disturbances are a complex process because voltage fluctuations affect different loads in different ways. Therefore, there is…
Superconducting qubits are an attractive platform for quantum computing since they have demonstrated high-fidelity quantum gates and extensibility to modest system sizes. Nonetheless, an outstanding challenge is stabilizing their…
The efficiency statistics of a small thermodynamic machine has been recently investigated assuming that the total dissipation was a linear combination of two currents: the input and output currents. Here, we relax this standard assumption…
It is demonstrated that in low multiplicity sample, the increase of the fluctuation of event-factorial-moments with the diminishing of phase space scale, called ``erraticity'', are dominated by the statistical fluctuations. The erraticity…
Computers are deterministic dynamical systems (CHAOS 19:033124, 2009). Among other things, that implies that one should be able to use deterministic forecast rules to predict their behavior. That statement is sometimes-but not always-true.…