Related papers: Statistical significance revisited
Which type of statistical uncertainty -- statistical (in)significance with a p-value, or a Bayesian probability -- enables people to see the continuous nature of uncertainty more clearly in a policymaking context? An original survey…
How can we draw trustworthy scientific conclusions? One criterion is that a study can be replicated by independent teams. While replication is critically important, it is arguably insufficient. If a study is biased for some reason and other…
We examine the extent to which random samplings from the values of a random set, determine the distribution of the random set itself. We also comment on how, given the statistics of the sampling, to detect the distribution. Several methods…
Results of simulation studies evaluating the performance of statistical methods are often considered actionable and thus can have a major impact on the way empirical research is implemented. However, so far there is limited evidence about…
Since its introduction by Fisher, the method of hypothesis testing that relies on computing error probabilities has witnessed several developments. Perhaps the most significant development was the seminal contributions of Neyman and Pearson…
Parametric statistical methods play a central role in analyzing risk through its underlying frequency and severity components. Given the wide availability of numerical algorithms and high-speed computers, researchers and practitioners often…
This paper considers the joint distribution of elements of a random sample and an order statistic of the same sample. \ The motivation for this work stems from the important problem in reliability analysis, to estimate the number of…
Simulation studies are computer experiments that involve creating data by pseudorandom sampling. The key strength of simulation studies is the ability to understand the behaviour of statistical methods because some 'truth' (usually some…
The influential claim that most published results are false raised concerns about the trustworthiness and integrity of science. Since then, there have been numerous attempts to examine the rate of false-positive results that have failed to…
Importance sampling is a well developed method in statistics. Given a random variable $X$, the problem of estimating its expected value $\mu$ is addressed. The standard approach is to use the sample mean as an estimator $\bar x$. In…
This paper describes Simpson's paradox, and explains its serious implications for randomised control trials. In particular, we show that for any number of variables we can simulate the result of a controlled trial which uniformly points to…
We define some appropriate statistical quantities that indicate the physical significance (reliability) of confidence intervals in the framework of both Frequentist and Bayesian statistical theories. We consider the expectation value of the…
Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning. Previous work has focused mainly on bounding either the expected loss of a predictor or the probability that an…
The basic idea of importance sampling is to use independent samples from a proposal measure in order to approximate expectations with respect to a target measure. It is key to understand how many samples are required in order to guarantee…
Adequate sampling is essential for the well-functioning of a market surveillance system. As small as possible statistically significant sample size is the main factor that determines the costs of market surveillance actions. This paper…
Computational method for statistical measures of reliability, confidence, and assurance are available for infinite population size. If the population size is finite and small compared to the number of samples tested, these computational…
The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In this paper, we make the scientific case for systematically…
This paper considers how to measure the magnitude of the sum of independent random variables in several ways. We give a formula for the tail distribution for sequences that satisfy the so called Levy property. We then give a connection…
This paper addresses the statistical significance of structures in random data: Given a set of vectors and a measure of mutual similarity, how likely does a subset of these vectors form a cluster with enhanced similarity among its elements?…
Medical and population health science researchers frequently make ambiguous statements about whether they believe their study sample or results are "representative" of some (implicit or explicit) target population. Here, we provide a…