Related papers: P values, confidence intervals, or confidence leve…
Practical or scientific considerations often lead to selecting a subset of parameters as ``important.'' Inferences about those parameters often are based on the same data used to select them in the first place. That can make the reported…
We formalize the idea of probability distributions that lead to reliable predictions about some, but not all aspects of a domain. The resulting notion of `safety' provides a fresh perspective on foundational issues in statistics, providing…
In their recent comment, published in Nature, Jeffrey T.Leek and Roger D.Peng discuss how P-values are widely abused in null hypothesis significance testing . We agree completely with them and in this short comment we discuss the importance…
In meta analysis, multiple hypothesis testing and many other methods, p-values are utilized as inputs and assumed to be uniformly distributed over the unit interval under the null hypotheses. If data used to generate p-values have discrete…
The Full Bayesian Significance Test (FBST) for precise hypotheses was presented by Pereira and Stern (1999) as a Bayesian alternative instead of the traditional significance test based on p-value. The FBST uses the evidence in favor of the…
We consider a general regression model, without a scale parameter. Our aim is to construct a confidence interval for a scalar parameter of interest $\theta$ that utilizes the uncertain prior information that a distinct scalar parameter…
In this paper, we are concerned with attributing meaning to the results of a Bayesian analysis for a problem which is sufficiently complex that we are unable to assert a precise correspondence between the expert probabilistic judgements of…
Turing's estimator allows one to estimate the probabilities of outcomes that either do not appear or only rarely appear in a given random sample. We perform a simulation study to understand the finite sample performance of several related…
We present a (selective) review of recent frequentist high-dimensional inference methods for constructing $p$-values and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which…
Conformal prediction is a powerful framework for distribution-free uncertainty quantification. The standard approach to conformal prediction relies on comparing the ranks of prediction scores: under exchangeability, the rank of a future…
Adaptive experiments use preliminary analyses of the data to inform further course of action and are commonly used in many disciplines including medical and social sciences. Because the null hypothesis and experimental design are…
We examine a new approach to modeling uncertainty based on plausibility measures, where a plausibility measure just associates with an event its plausibility, an element is some partially ordered set. This approach is easily seen to…
Suppose the lifetime of a large sample of batteries in routine use is measured. A confidence interval is computed to 394 plus/minus 1.96 times 4.6 days. The standard interpretation is that if we repeatedly draw samples and compute…
Normalised citation counts are routinely used to assess the average impact of research groups or nations. There is controversy over whether confidence intervals for them are theoretically valid or practically useful. In response, this…
This article develops $p$-values for evaluating means of normal populations that make use of indirect or prior information. A $p$-value of this type is based on a biased test statistic that is optimal on average with respect to a…
Statistical analysis is often used to evaluate the evidence for or against scientific hypotheses, and various statistics (e.g., p-values, likelihood ratios, Bayes factors) are interpreted as measures of evidence strength. Here I consider…
As neural networks become more popular, the need for accompanying uncertainty estimates increases. There are currently two main approaches to test the quality of these estimates. Most methods output a density. They can be compared by…
An important component in deploying machine learning (ML) in safety-critic applications is having a reliable measure of confidence in the ML model's predictions. For a classifier $f$ producing a probability vector $f(x)$ over the candidate…
We present the expected values from p-value hacking as a choice of the minimum p-value among $m$ independents tests, which can be considerably lower than the "true" p-value, even with a single trial, owing to the extreme skewness of the…
In various high-energy physics contexts, such as neutrino-oscillation experiments, several assumptions underlying the typical asymptotic confidence interval construction are violated, such that one has to resort to computationally expensive…