Related papers: Randomized p-values for multiple testing and their…
Randomization tests are based on a re-randomization of existing data to gain data-dependent critical values that lead to exact hypothesis tests under special circumstances. However, it is not always possible to re-randomize data in…
Introductory statistical inference texts and courses treat the point estimation, hypothesis testing, and interval estimation problems separately, with primary emphasis on large-sample approximations. Here I present an alternative approach…
Background: The standard regulatory approach to assess replication success is the two-trials rule, requiring both the original and the replication study to be significant with effect estimates in the same direction. The sceptical p-value…
In large-scale hypothesis testing, computing exact $p$-values or $e$-values is often resource-intensive, creating a need for budget-aware inferential methods. We propose a general framework for active hypothesis testing that leverages…
This article explains, and discusses the merits of, three approaches for analyzing the certainty with which statistical results can be extrapolated beyond the data gathered. Sometimes it may be possible to use more than one of these…
In the field of multiple hypothesis testing, combining p-values represents a fundamental statistical method. The Cauchy combination test (CCT) (Liu and Xie, 2020) excels among numerous methods for combining p-values with powerful and…
For a given testing problem, let $U_1,...,U_n$ be individually valid and conditionally on the data i.i.d.\ P-variables (often called P-values). For example, the data could come in groups, and each $U_i$ could be based on subsampling just…
Generalizations to the permutation test are introduced to allow for situations in which the null model is not exchangeable. It is shown that the generalized permutation tests are exact, and a partial converse: that any test function that is…
We consider the problem of testing for a difference in means between clusters of observations identified via k-means clustering. In this setting, classical hypothesis tests lead to an inflated Type I error rate. To overcome this problem, we…
We put forward a novel calibration of p values, the "Adaptive Robust Lower Bound" (ARLB) which maps p values into approximations of posterior probabilities taking into account the effect of sample sizes. We build on the Robust Lower Bound…
Over the last decade, the Dip-test of unimodality has gained increasing interest in the data mining community as it is a parameter-free statistical test that reliably rates the modality in one-dimensional samples. It returns a so called…
Estimating the dependences between random variables, and ranking them accordingly, is a prevalent problem in machine learning. Pursuing frequentist and information-theoretic approaches, we first show that the p-value and the mutual…
Testing composite null hypotheses arises in various applications, such as mediation and replicability analyses. The problem becomes more challenging in high-throughput experiments where tens of thousands of features are examined…
So-called linear rank statistics provide a means for distribution-free (even in finite samples), yet highly flexible, two-sample testing in the setting of univariate random variables. Their flexibility derives from a choice of weights that…
A common problem in genetics is that of testing whether a set of highly dependent gene expressions differ between two populations, typically in a high-dimensional setting where the data dimension is larger than the sample size. Most…
In this article, we derive and compare methods to derive \textit{p}-values and sets of confidence intervals with strong control of the family-wise error rates and coverage for estimates of treatment effects in cluster randomised trials with…
Permutation tests are a distribution free way of performing hypothesis tests. These tests rely on the condition that the observed data are exchangeable among the groups being tested under the null hypothesis. This assumption is easily…
In this paper, we demonstrate that a new measure of evidence we developed called the Dempster-Shafer p-value which allow for insights and interpretations which retain most of the structure of the p-value while covering for some of the…
In this paper, we study randomized methods for feedback design of uncertain systems. The first contribution is to derive the sample complexity of various constrained control problems. In particular, we show the key role played by the…
We consider estimating the proportion of random variables for two types of composite null hypotheses: (i) the means of the random variables belonging to a non-empty, bounded interval; (ii) the means of the random variables belonging to an…