相关论文: A recipe for irreproducible results
We study a model of temporal voting where there is a fixed time horizon, and at each round the voters report their preferences over the available candidates and a single candidate is selected. Prior work has adapted popular notions of…
Human evaluation is critical for validating the performance of text-to-image generative models, as this highly cognitive process requires deep comprehension of text and images. However, our survey of 37 recent papers reveals that many works…
Mathematical models of complex social systems can enrich social scientific theory, inform interventions, and shape policy. From voting behavior to economic inequality and urban development, such models influence decisions that affect…
Reproducibility of computationally-derived scientific discoveries should be a certainty. As the product of several person-years' worth of effort, results -- whether disseminated through academic journals, conferences or exploited through…
A new formula for the probability that a standard Brownian motion stays between two linear boundaries is proved. A simple algorithm is deduced. Uniform precision estimates are computed. Different implementations have been made available…
We introduce a new model that mimics the strong and sudden effects induced by conformity in tightly interacting human societies. Such effects range from mere crowd phenomena to dramatic political turmoil. The model is a modified version of…
Subtraction of ``accidentals'' in Einstein-Podolsky-Rosen experiments frequently changes results compatible with local realism into ones that appear to demonstrate non-locality. The validity of the procedure depends on the unproven…
This letter concerns the reliability of coupled oscillator networks in response to fluctuating inputs. Reliability means that (following a transient) an input elicits identical responses upon repeated presentations, regardless of the…
Within the framework of the hypothesis offered by authors about a complex-valued nature of physical quantities the stability of basic equations of the classical physics concerning complex-valued perturbations of parameters and boundary…
We derive analytic formulas to reconstruct particle-averaged quantities from experimental results that suffer from the efficiency loss of particle measurements. These formulas are derived under the assumption that the probabilities of…
In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is…
We examine stochastic processes that are used to model nonequilibrium processes (e.g, pulling RNA or dragging colloids) and so deliberately violate detailed balance. We argue that by combining an information-theoretic measure of…
Reproducibility is essential to reliable scientific discovery in high-throughput experiments. In this work we propose a unified approach to measure the reproducibility of findings identified from replicate experiments and identify putative…
Context: Software Engineering (SE) experiments suffer from threats to validity that may impact their results. Replication allows researchers building on top of previous experiments' weaknesses and increasing the reliability of the findings.…
Recent advancements in machine learning have emphasized the need for transparency in model predictions, particularly as interpretability diminishes when using increasingly complex architectures. In this paper, we propose leveraging…
We focus on the dynamics of a Brownian particle whose mass fluctuates. First we show that the behaviour is similar to that of a Brownian particle moving in a fluctuating medium, as studied by Beck [Phys. Rev. Lett. 87 (2001) 180601]. By…
Attempts to replicate probabilistic reasoning in expert systems have typically overlooked a critical ingredient of that process. Probabilistic analysis typically requires extensive judgments regarding interdependencies among hypotheses and…
Reproducibility is an ideal that no researcher would dispute "in the abstract", but when aspirations meet the cold hard reality of the academic grind, reproducibility often "loses out". In this essay, I share some personal experiences…
Rapid advances in computing technology over the past few decades have spurred two extraordinary phenomena in science: large-scale and high-throughput data collection coupled with the creation and implementation of complex statistical…
A key trait of stochastic optimizers is that multiple runs of the same optimizer in attempting to solve the same problem can produce different results. As a result, their performance is evaluated over several repeats, or runs, on the…