Related papers: Conditional Equivalence Testing: an alternative re…
We propose novel kernel-based tests for assessing the equivalence between distributions. Traditional goodness-of-fit testing is inappropriate for concluding the absence of distributional differences, because failure to reject the null…
This paper explores Bayesian estimation for categorical data, focusing on simple yet effective models that provide a foundation for applying more advanced methods accurately and reliably in real-world applications. We begin by revisiting…
This paper explores conditions of existence of different types of consistent tests. New links of these types of consistency are also established. The existence of discernible (strong consistent) tests follows from the existence of pointwise…
Colocalization analyses assess whether two traits are affected by the same or distinct causal genetic variants in a single gene region. A class of Bayesian colocalization tests are now routinely used in practice; for example, for genetic…
Recently, conditional average treatment effect (CATE) estimation has been attracting much attention due to its importance in various fields such as statistics, social and biomedical sciences. This study proposes a partially linear…
In prior work we have introduced an asymptotic threshold of sufficient randomness for causal inference from observational data. In this paper we extend that prior work in three main ways. First, we show how to empirically estimate a lower…
Large language models often improve reasoning by sampling multiple outputs and aggregating their final answers, but precise and efficient control of error levels remains a challenging task. In particular, deciding when to stop sampling…
This paper proposes self-normalized tests for multistep conditional predictive ability in forecast comparison. By normalizing the sample mean of the transformed loss differential using functionals of its cumulative sum (CUSUM) process,…
Null Hypothesis Significance Testing (NHST) has long been of central importance to psychology as a science, guiding theory development and underlying the application of evidence-based intervention and decision-making. Recent years, however,…
We investigate the discrimination of two candidates of an unknown parameter in quantum systems with continuous weak measurement, inspired by the application of hypothesis testing in distinguish-ing two Hamiltonians [Kiilerich and M{\o}lmer,…
When interpreting A/B tests, we typically focus only on the statistically significant results and take them by face value. This practice, termed post-selection inference in the statistical literature, may negatively affect both point…
In the field of multiple hypothesis testing, combining p-values represents a fundamental statistical method. The Cauchy combination test (CCT) (Liu and Xie, 2020) excels among numerous methods for combining p-values with powerful and…
We study hybrid control trials (HCTs), in which a randomized controlled trial (RCT) is augmented with external control patients. Existing approaches for HCTs typically assume conditional exchangeability of the concurrent and external…
Time Series Anomaly Detection metrics serve as crucial tools for model evaluation. However, existing metrics suffer from several limitations: insufficient discriminative power, strong hyperparameter dependency, sensitivity to perturbations,…
We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity. The alternatives are stationary self-exciting point processes. We…
Foundation models often generate unreliable answers, while heuristic uncertainty estimators fail to fully distinguish correct from incorrect outputs, causing users to accept erroneous answers without any statistical guarantee. We address…
The e-value is swiftly rising in prominence in many applications of hypothesis testing and multiple testing, yet its relationship to classical testing theory remains elusive. We unify e-values and classical testing into a single 'continuous…
Contrast-Consistent Search (CCS) is an unsupervised probing method able to test whether large language models represent binary features, such as sentence truth, in their internal activations. While CCS has shown promise, its two-term…
In most prediction and estimation situations, scientists consider various statistical models for the same problem, and naturally want to select amongst the best. Hansen et al. (2011) provide a powerful solution to this problem by the…
Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, existing scaling methods have key limitations: parallel…