Carefree multiple testing with e-processes
Abstract
E-processes enable hypothesis testing with ongoing data collection while maintaining Type I error control. However, when testing multiple hypotheses simultaneously, current -value based multiple testing methods such as e-BH are not invariant to the order in which data are gathered for the different -processes. This can lead to undesirable situations, e.g., where a hypothesis rejected at time is no longer rejected at time after choosing to gather more data for one or more -processes unrelated to that hypothesis. We argue that multiple testing methods should always work with suprema of -processes. We provide an example to illustrate that e-BH does not control the FDR, at level when applied to suprema of -processes. From the same example we see that the FWER is not controlled with averaging, and also closed e-BH does not control the FDR. We show that adjusters can be used to ensure FDR-sup control with e-BH under arbitrary dependence.
Keywords
Cite
@article{arxiv.2501.19360,
title = {Carefree multiple testing with e-processes},
author = {Yury Tavyrikov and Jelle J. Goeman and Rianne de Heide},
journal= {arXiv preprint arXiv:2501.19360},
year = {2025}
}