English
Related papers

Related papers: To how many simultaneous hypothesis tests can norm…

200 papers

For classification models based on neural networks, the maximum predicted class probability is often used as a confidence score. This score rarely predicts well the probability of making a correct prediction and requires a post-processing…

Machine Learning · Computer Science 2024-11-07 Adrien LeCoz , Stéphane Herbin , Faouzi Adjed

One of the most commonly used methods for forming confidence intervals for statistical inference is the empirical bootstrap, which is especially expedient when the limiting distribution of the estimator is unknown. However, despite its…

Statistics Theory · Mathematics 2020-11-24 Morgane Austern , Vasilis Syrgkanis

We consider large-scale studies in which thousands of significance tests are performed simultaneously. In some of these studies, the multiple testing procedure can be severely biased by latent confounding factors such as batch effects and…

Methodology · Statistics 2016-06-21 Jingshu Wang , Qingyuan Zhao , Trevor Hastie , Art B. Owen

Spectral analysis plays a crucial role in high-dimensional statistics, where determining the asymptotic distribution of various spectral statistics remains a challenging task. Due to the difficulties of deriving the analytic form, recent…

Statistics Theory · Mathematics 2025-04-02 Guoyu Zhang , Dandan Jiang , Fang Yao

Current statistical inference problems in areas like astronomy, genomics, and marketing routinely involve the simultaneous testing of thousands -- even millions -- of null hypotheses. For high-dimensional multivariate distributions, these…

Methodology · Statistics 2017-04-25 Weixin Cai , Nima S. Hejazi , Alan E. Hubbard

Basket trials are increasingly used for the simultaneous evaluation of a new treatment in various patient subgroups under one overarching protocol. We propose a Bayesian approach to sample size determination in basket trials that permit…

Methodology · Statistics 2022-09-02 Haiyan Zheng , Michael J. Grayling , Pavel Mozgunov , Thomas Jaki , James M. S. Wason

We study the scaling of classification error rates with respect to the size of the training dataset. In contrast to classical results where rates are minimax optimal for a problem class, this work starts with the empirical observation that,…

Machine Learning · Statistics 2025-06-04 Pengkun Yang , Jingzhao Zhang

Consider two problems about an unknown probability distribution $p$: 1. How many samples from $p$ are required to test if $p$ is supported on $n$ elements or not? Specifically, given samples from $p$, determine whether it is supported on at…

Data Structures and Algorithms · Computer Science 2026-05-27 Renato Ferreira Pinto , Nathaniel Harms

We show that top-down decision tree learning heuristics are amenable to highly efficient learnability estimation: for monotone target functions, the error of the decision tree hypothesis constructed by these heuristics can be estimated with…

Machine Learning · Computer Science 2020-11-04 Guy Blanc , Neha Gupta , Jane Lange , Li-Yang Tan

We propose a new approach to the problem of high-dimensional multivariate ANOVA via bootstrapping max statistics that involve the differences of sample mean vectors. The proposed method proceeds via the construction of simultaneous…

Methodology · Statistics 2021-04-20 Zhenhua Lin , Miles E. Lopes , Hans-Georg Müller

As the most important tool to provide high-level evidence-based medicine, researchers can statistically summarize and combine data from multiple studies by conducting meta-analysis. In meta-analysis, mean differences are frequently used…

Methodology · Statistics 2018-01-30 Dehui Luo , Xiang Wan , Jiming Liu , Tiejun Tong

In this paper we address the problem of performing statistical inference for large scale data sets i.e., Big Data. The volume and dimensionality of the data may be so high that it cannot be processed or stored in a single computing node. We…

Methodology · Statistics 2016-04-20 Shahab Basiri , Esa Ollila , Visa Koivunen

High complexity models are notorious in machine learning for overfitting, a phenomenon in which models well represent data but fail to generalize an underlying data generating process. A typical procedure for circumventing overfitting…

Machine Learning · Statistics 2025-03-11 James Schmidt

Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not…

Machine Learning · Statistics 2019-11-01 Jayaraman J. Thiagarajan , Bindya Venkatesh , Deepta Rajan

This work presents a statistically principled method for estimating the required number of instances in the experimental comparison of multiple algorithms on a given problem class of interest. This approach generalises earlier results by…

Methodology · Statistics 2019-08-06 Felipe Campelo , Elizabeth F. Wanner

It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the…

Computer Vision and Pattern Recognition · Computer Science 2022-07-25 Tom Joy , Francesco Pinto , Ser-Nam Lim , Philip H. S. Torr , Puneet K. Dokania

Modern problems in statistics tend to include estimators of high computational complexity and with complicated distributions. Statistical inference on such estimators usually relies on asymptotic normality assumptions, however, such…

Methodology · Statistics 2016-12-08 Eyal Fisher , Regev Schweiger , Saharon Rosset

A/B testing has become the cornerstone of decision-making in online markets, guiding how platforms launch new features, optimize pricing strategies, and improve user experience. In practice, we typically employ the pairwise $t$-test to…

Machine Learning · Statistics 2025-10-29 Junpeng Gong , Chunkai Wang , Hao Li , Jinyong Ma , Haoxuan Li , Xu He

Simultaneous inference allows for the exploration of data while deciding on criteria for proclaiming discoveries. It was recently proved that all admissible post-hoc inference methods for true discoveries must employ closed testing. In this…

Statistics Theory · Mathematics 2022-03-24 Jinjin Tian , Xu Chen , Eugene Katsevich , Jelle Goeman , Aaditya Ramdas

Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty…