Related papers: Test statistics and p-values
Experimental studies are a cornerstone of Machine Learning (ML) research. A common and often implicit assumption is that the study's results will generalize beyond the study itself, e.g., to new data. That is, repeating the same study under…
Two-sided statistical tests and p-values are well defined only when the test statistic in question has a symmetric distribution. A new two-sided p-value called conditional p-value $P_C$ is introduced here. It is closely related to the…
We study a large-scale one-sided multiple testing problem in which test statistics follow normal distributions with unit variance, and the goal is to identify signals with positive mean effects. A conventional approach is to compute…
The aim of this article is to promote the use of probabilistic methods in the study of problems in mathematical general relativity. Two new and simple singularity theorems, whose features are different from the classical singularity…
Probabilistic metrology attempts to improve parameter estimation by occasionally reporting an excellent estimate and the rest of the time either guessing or doing nothing at all. Here we show that probabilistic metrology can never improve…
It is observed that for testing between simple hypotheses where the cost of Type I and Type II errors can be quantified, it is better to let the optimization choose the test size.
We explain the concept of p-values presupposing only rudimentary probability theory. We also use the occasion to introduce the notion of p-function, so that p-values are values of a p-function. The explanation is restricted to the discrete…
Our purpose, is to put forward a change in the paradigm of testing by generalizing a very natural idea exposed by Morris DeGroot (1975) aiming to an approach that is attractive to all schools of statistics, in a procedure better suited for…
$P$-values that are derived from continuously distributed test statistics are typically uniformly distributed on $(0,1)$ under least favorable parameter configurations (LFCs) in the null hypothesis. Conservativeness of a $p$-value $P$…
The idea of fully accepting statements when the evidence has rendered them probable enough faces a number of difficulties. We leave the interpretation of probability largely open, but attempt to suggest a contextual approach to full belief.…
We provide a fully statistical analysis of the results of a Bell test beyond mean values. This is possible in a practical scheme where all the observables involved in the test are simultaneously measured at the expense of unavoidably…
The asymptotically optimal hypothesis testing problem with the general sources as the null and alternative hypotheses is studied under exponential-type error constraints on the first kind of error probability. Our fundamental philosophy in…
The limitation of permutation tests is that they assume exchangeability. It is shown that in generalized linear models one can construct permutation tests from score statistics in particular cases. When under the null hypothesis the…
As a convention, p-value is often computed in frequentist hypothesis testing and compared with the nominal significance level of 0.05 to determine whether or not to reject the null hypothesis. The smaller the p-value, the more significant…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
Bayesian, classical, and extended maximum likelihood approaches to estimation of upper limits in experiments with small numbers of signal events are surveyed. The discussion covers only experiments whose outcomes are well described by a…
In this paper, we demonstrate that a new measure of evidence we developed called the Dempster-Shafer p-value which allow for insights and interpretations which retain most of the structure of the p-value while covering for some of the…
In this note, we propose a robustified analogue of the conventional Student $t$-test statistic. The proposed statistic is easy to implement and thus practically useful. We also show that it is a pivotal quantity and converges to a standard…
This paper is concerned with the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all of modern information processing. Accounting for constraints, however, is paramount to…
A family of consistent tests, derived from a characterization of the probability generating function, is proposed for assessing Poissonity against a wide class of count distributions, which includes some of the most frequently adopted…