Related papers: Too Many, Too Improbable: testing joint hypotheses…
We seek to design novel multiple testing procedures, which take into account a relevant notion of ''power'' or true discovery on the one hand, and allow computationally efficient test design and application on the other. Towards this end we…
The sequential multiple testing problem is considered under two generalized error metrics. Under the first one, the probability of at least $k$ mistakes, of any kind, is controlled. Under the second, the probabilities of at least $k_1$…
In this work, we address the question of how to enhance signal-agnostic searches by leveraging multiple testing strategies. Specifically, we consider hypothesis tests relying on machine learning, where model selection can introduce a bias…
The maximum type-I and type-II error exponents associated with the newly introduced almost-fixed-length hypothesis testing is characterized. In this class of tests, the decision-maker declares the true hypothesis almost always after…
High complexity models are notorious in machine learning for overfitting, a phenomenon in which models well represent data but fail to generalize an underlying data generating process. A typical procedure for circumventing overfitting…
The closure principle is fundamental in multiple testing and has been used to derive many efficient procedures with familywise error rate control. However, it is often unsuitable for modern research, which involves flexible multiple testing…
Conjoint analysis is a popular experimental design used to measure multidimensional preferences. Researchers examine how varying a factor of interest, while controlling for other relevant factors, influences decision-making. Currently,…
In many large multiple testing problems the hypotheses are divided into families. Given the data, families with evidence for true discoveries are selected, and hypotheses within them are tested. Neither controlling the error-rate in each…
Empirical economic studies often involve multiple propositions or hypotheses, with researchers aiming to assess both the collective and individual evidence against these propositions or hypotheses. To rigorously assess this evidence,…
Consider the multiple testing problem of testing null hypotheses $H_1,...,H_s$. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate ($\mathit{FWER}$),…
We discuss a general approach to handling "multiple hypotheses" testing in the case when a particular hypothesis states that the vector of parameters identifying the distribution of observations belongs to a convex compact set associated…
It is often of interest to test a global null hypothesis using multiple, possibly dependent $p$-values by combining their strengths while controlling the type-I error. Recently, several heavy-tailed combination tests, such as the harmonic…
In contemporary research, data scientists often test an infinite sequence of hypotheses $H_1,H_2,\ldots$ one by one, and are required to make real-time decisions without knowing the future hypotheses or data. In this paper, we consider such…
Multiple testing problems arising in modern scientific applications can involve simultaneously testing thousands or even millions of hypotheses, with relatively few true signals. In this paper, we consider the multiple testing problem where…
Particularly in genomics, but also in other fields, it has become commonplace to undertake highly multiple Student's $t$-tests based on relatively small sample sizes. The literature on this topic is continually expanding, but the main…
Multiple hypothesis testing often involves composite nulls, i.e., nulls that are associated with two or more distributions. In many cases, it is reasonable to assume that there is a prior distribution on the distributions despite it is…
The problem of multi-hypothesis testing with controlled sensing of observations is considered. The distribution of observations collected under each control is assumed to follow a single-parameter exponential family distribution. The goal…
This paper is a review of the popular Benjamini Hochberg Method and other related useful methods of Multiple Hypothesis testing. This is written with the purpose of serving a short but complete easy to understand review of the main article…
Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest, a sequential algorithm which gives, with arbitrarily high probability,…
Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied.…