Related papers: Decision-theoretic justifications for Bayesian hyp…
Hypothesis testing is a central statistical method in psychological research and the cognitive sciences. While the problems of null hypothesis significance testing (NHST) have been debated widely, few attractive alternatives exist. In this…
Usually one compares the accuracy of two competing classifiers via null hypothesis significance tests (nhst). Yet the nhst tests suffer from important shortcomings, which can be overcome by switching to Bayesian hypothesis testing. We…
While observational data are routinely used to estimate causal effects of biomedical treatments, doing so requires special methods to adjust for observed confounding. These methods invariably rely on untestable statistical and causal…
Test-time adaptation (TTA) methods improve model performance under distribution shift but lack formal guarantees connecting shift magnitude to prediction reliability. We develop a PAC-Bayesian framework yielding generalization bounds…
A statistical test can be seen as a procedure to produce a decision based on observed data, where some decisions consist of rejecting a hypothesis (yielding a significant result) and some do not, and where one controls the probability to…
Testing the equality of two proportions is a common procedure in science, especially in medicine and public health. In these domains it is crucial to be able to quantify evidence for the absence of a treatment effect. Bayesian hypothesis…
Statistical learning theory is the foundation of machine learning, providing theoretical bounds for the risk of models learned from a (single) training set, assumed to issue from an unknown probability distribution. In actual deployment,…
The validity of classical hypothesis testing requires the significance level $\alpha$ be fixed before any statistical analysis takes place. This is a stringent requirement. For instance, it prohibits updating $\alpha$ during (or after) an…
The last decade witnessed an explosion in the availability of data for operations research applications. Motivated by this growing availability, we propose a novel schema for utilizing data to design uncertainty sets for robust optimization…
Consider a random vector $(X,Y)$ and let $m(x)=E(Y|X=x)$. We are interested in testing $H_0:m\in {\cal M}_{\Theta,{\cal G}}=\{\gamma(\cdot,\theta,g):\theta \in \Theta,g\in {\cal G}\}$ for some known function $\gamma$, some compact set…
In the hypothesis testing framework, p-value is often computed to determine rejection of the null hypothesis or not. On the other hand, Bayesian approaches typically compute the posterior probability of the null hypothesis to evaluate its…
A standard practice in statistical hypothesis testing is to mention the p-value alongside the accept/reject decision. We show the advantages of mentioning an e-value instead. With p-values, it is not clear how to use an extreme observation…
Clustering is part of unsupervised analysis methods that consist in grouping samples into homogeneous and separate subgroups of observations also called clusters. To interpret the clusters, statistical hypothesis testing is often used to…
We generalize standard credal set models for imprecise probabilities to include higher order credal sets -- confidences about confidences. In doing so, we specify how an agent's higher order confidences (credal sets) update upon observing…
Two recently introduced model based bias corrected estimators for proportion of true null hypotheses ($\pi_0$) under multiple hypotheses testing scenario have been restructured for exponentially distributed random observations available for…
In the problem of composite hypothesis testing, identifying the potential uniformly most powerful (UMP) unbiased test is of great interest. Beyond typical hypothesis settings with exponential family, it is usually challenging to prove the…
The non-convexity and intractability of distributionally robust chance constraints make them challenging to cope with. From a data-driven perspective, we propose formulating it as a robust optimization problem to ensure that the…
Necessary and sufficient conditions of uniform consistency are explored. A hypothesis is simple. Nonparametric sets of alternatives are bounded convex sets in $\mathbb{L}_p$, $p >1$ with "small" balls deleted. The "small" balls have the…
We introduce a new conservative test for quantifying the consistency of two or more datasets. The test is based on the Bayesian answer to the question, ``How much more probable is it that all my data were generated from the same model…
In this paper, our interest is in the problem of simultaneous hypothesis testing when the test statistics corresponding to the individual hypotheses are possibly correlated. Specifically, we consider the case when the test statistics…