Related papers: A Statistical Significance Simulation Study for th…
We describe a method for estimation of the discovery potential on new physics in planned experiments. The effective significance of signal for given probability of observation is proposed for planned experiments instead of the usual…
Estimating the probability distribution 'q' governing the behaviour of a certain variable by sampling its value a finite number of times most typically involves an error. Successive measurements allow the construction of a histogram, or…
There are phenomena that cannot be measured without subjective testing. However, subjective testing is a complex issue with many influencing factors. These interplay to yield either precise or incorrect results. Researchers require a tool…
Statistical mechanics relies on the complete though probabilistic description of a system in terms of all the microscopic variables. Its object is to derive therefrom static and dynamic properties involving some reduced set of variables.…
Verifying that a statistically significant result is scientifically meaningful is not only good scientific practice, it is a natural way to control the Type I error rate. Here we introduce a novel extension of the p-value - a…
Consistency, defined as the requirement that a series of measurements of the same project carried out by different raters using the same method should produce similar results, is one of the most important aspects to be taken into account in…
Within the framework of likelihood-based statistical tests for high energy physics measurements, we derive generalized expressions for estimating the statistical significance of discovery using the asymptotic approximations of Wilks and…
A large deviation function mathematically characterizes the statistical property of atypical events. Recently, in non-equilibrium statistical mechanics, large deviation functions have been used to describe universal laws such as the…
Judging the significance and reproducibility of quantitative research requires a good understanding of relevant uncertainties, but it is often unclear how well these have been evaluated and what they imply. Reported scientific uncertainties…
Statistical science (as opposed to mathematical statistics) involves far more than probability theory, for it requires realistic causal models of data generators - even for purely descriptive goals. Statistical decision theory requires more…
We study the task of retrieving relevant experiments given a query experiment. By experiment, we mean a collection of measurements from a set of `covariates' and the associated `outcomes'. While similar experiments can be retrieved by…
There are various approaches to the problem of how one is supposed to conduct a statistical analysis. Different analyses can lead to contradictory conclusions in some problems so this is not a satisfactory state of affairs. It seems that…
This paper is about how we study statistical methods. As an example, it uses the random regressions model, in which the intercept and slope of cluster-specific regression lines are modeled as a bivariate random effect. Maximizing this…
The Poisson probability distribution is frequently encountered in physical science measurements. In spite of the simplicity and familiarity of this distribution, there is considerable confusion among physicists concerning the description of…
Analysis of experimental data must sometimes deal with abrupt changes in the distribution of measured values. Setting upper limits on signals usually involves a veto procedure that excludes data not described by an assumed statistical…
Traditional statistical inference considers relatively small data sets and the corresponding theoretical analysis focuses on the asymptotic behavior of a statistical estimator when the number of samples approaches infinity. However, many…
In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a "replication success". Here we discuss the logical problems with this approach:…
Measuring science is based on comparing articles to similar others. However, keyword-based groups of thematically similar articles are dominantly small. These small sizes keep the statistical errors of comparisons high. With the growing…
In a recent opinion article, Muff et al. recapitulate well-known objections to the Neyman-Pearson Null-Hypothesis Significance Testing (NHST) framework and call for reforming our practices in statistical reporting. We agree with them on…
Statistical dependence between hypotheses poses a significant challenge to the stability of large scale multiple hypotheses testing. Ignoring it often results in an unacceptably large spread in the false positive proportion even though the…