Related papers: Cross-validation Confidence Intervals for Test Err…

We introduce a rigorous and sensitive significance test for hyperuniformity that yields reliable results even from a single sample. Our approach is based on a detailed analysis of the empirical Fourier transform of a stationary point…

Deep learning models have significantly improved prediction accuracy in various fields, gaining recognition across numerous disciplines. Yet, an aspect of deep learning that remains insufficiently addressed is the assessment of prediction…

Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and…

We consider a one-dimensional Gaussian process having exponential covariance function. Under fixed-domain asymptotics, we prove the strong consistency and asymptotic normality of a cross validation estimator of the microergodic covariance…

Confidence calibration for classification models is vital in safety-critical decision-making scenarios and has received extensive attention. General confidence calibration methods assume training and test data are independent and…

Least absolute shrinkage and selection operator or Lasso is one of the widely used regularization methods in regression. Statisticians usually implement Lasso in practice by choosing the penalty parameter in a data-dependent way, the most…

For linear models that may have asymmetric errors, we study variable selection by cross-validation. The data are split into training and validation sets, with the number of observations in the validation set much larger than in the training…

Robust estimators for linear regression require non-convex objective functions to shield against adverse affects of outliers. This non-convexity brings challenges, particularly when combined with penalization in high-dimensional settings.…

Recent advances in statistical theory, together with advances in the computational power of computers, provide alternative methods to do mass-univariate hypothesis testing in which a large number of univariate tests, can be properly used to…

Given an IID sample from a positive distribution, we provide a method for constructing rigorous finite sample lower confidence bounds for the expectation of the distribution. The method is based on constructing rigorous confidence regions…

The cross entropy loss is widely used due to its effectiveness and solid theoretical grounding. However, as training progresses, the loss tends to focus on hard to classify samples, which may prevent the network from obtaining gains in…

We identify the critical deviation scale governing Bayesian evidence accumulation in regular parametric testing. Under integrated Bayes risk with zero-one loss, the risk-optimal rejection boundary lies in a moderate deviation regime, with a…

Calibrated probability outputs of trained classifiers are increasingly used as inputs to downstream regression estimands such as effects, prevalences, or disparities for a latent group observed only on a small labelled subset. A standard…

We introduce a new cross-validation method based on an equicorrelated Gaussian randomization scheme. Our method is well-suited for problems where sample splitting is infeasible, either because the data violate the assumption of independent…

It is well known that if the power spectral density of a continuous time stationary stochastic process does not have a compact support, data sampled from that process at any uniform sampling rate leads to biased and inconsistent spectrum…

Reliable probabilistic primality tests are fundamental in public-key cryptography. In adversarial scenarios, a composite with a high probability of passing a specific primality test could be chosen. In such cases, we need worst-case error…

It is well known that the asymptotic variance of sample quantiles can be reduced under heterogeneity relative to the i.i.d. setting. However, asymptotically correct confidence intervals for quantiles are not yet available. We propose a…

Inference for prediction errors is critical in time series forecasting pipelines. However, providing statistically meaningful uncertainty intervals for prediction errors remains relatively under-explored. Practitioners often resort to…

Practical model building processes are often time-consuming because many different models must be trained and validated. In this paper, we introduce a novel algorithm that can be used for computing the lower and the upper bounds of model…

Recently, there has been substantial interest in statistical guarantees for cross-validation (CV) methods of uncertainty quantification in statistical learning (cf. Barber et al. 2021a, Liang and Barber 2024, Steinberger and Leeb 2023).…