English

Carefree multiple testing with e-processes

Statistics Theory 2025-07-18 v2 Methodology Statistics Theory

Abstract

E-processes enable hypothesis testing with ongoing data collection while maintaining Type I error control. However, when testing multiple hypotheses simultaneously, current ee-value based multiple testing methods such as e-BH are not invariant to the order in which data are gathered for the different ee-processes. This can lead to undesirable situations, e.g., where a hypothesis rejected at time tt is no longer rejected at time t+1t+1 after choosing to gather more data for one or more ee-processes unrelated to that hypothesis. We argue that multiple testing methods should always work with suprema of ee-processes. We provide an example to illustrate that e-BH does not control the FDR, at level α\alpha when applied to suprema of ee-processes. From the same example we see that the FWER is not controlled with averaging, and also closed e-BH does not control the FDR. We show that adjusters can be used to ensure FDR-sup control with e-BH under arbitrary dependence.

Keywords

Cite

@article{arxiv.2501.19360,
  title  = {Carefree multiple testing with e-processes},
  author = {Yury Tavyrikov and Jelle J. Goeman and Rianne de Heide},
  journal= {arXiv preprint arXiv:2501.19360},
  year   = {2025}
}