Related papers: A frequentist two-sample test based on Bayesian mo…
Bayes Factors, the Bayesian tool for hypothesis testing, are receiving increasing attention in the literature. Compared to their frequentist rivals ($p$-values or test statistics), Bayes Factors have the conceptual advantage of providing…
We provide a brief overview of both Bayes and classical model selection. We argue tentatively that model selection has at least two major goals, that of finding the correct model or predicting well, and that in general both these goals may…
Two-sample tests evaluate whether two samples are realizations of the same distribution (the null hypothesis) or two different distributions (the alternative hypothesis). We consider a new setting for this problem where sample features are…
A common task in high-throughput biology is to test for differences in means between two samples across thousands of features (e.g., genes or proteins), often with only a handful of replicates per sample. Moderated t-tests handle this…
Testing the equality of two proportions is a common procedure in science, especially in medicine and public health. In these domains it is crucial to be able to quantify evidence for the absence of a treatment effect. Bayesian hypothesis…
A new computation method of frequentist $p$-values and Bayesian posterior probabilities based on the bootstrap probability is discussed for the multivariate normal model with unknown expectation parameter vector. The null hypothesis is…
Design of experiments has traditionally relied on the frequentist hypothesis testing framework where the optimal size of the experiment is specified as the minimum sample size that guarantees a required level of power. Sample size…
Statistics comes in two main flavors: frequentist and Bayesian. For historical and technical reasons, frequentist statistics have traditionally dominated empirical data analysis, and certainly remain prevalent in empirical software…
Analogues of the frequentist chi-square and F tests are proposed for testing goodness-of-fit and consistency for Bayesian models. Simple examples exhibit these tests' detection of inconsistency between consecutive experiments with identical…
A common concern with Bayesian methodology in scientific contexts is that inferences can be heavily influenced by subjective biases. As presented here, there are two types of bias for some quantity of interest: bias against and bias in…
Statistics comes in two main flavors: frequentist and Bayesian. For historical and technical reasons, frequentist statistics has dominated data analysis in the past; but Bayesian statistics is making a comeback at the forefront of science.…
We present the fundamental ideas underlying statistical hypothesis testing using the frequentist framework. We begin with a simple example that builds up the one-sample t-test from the beginning, explaining important concepts such as the…
The two statistical methods, namely the frequentist and the Bayesian methods, are both commonly used for probabilistic inference in many scientific situations. However, it is not straightforward to interpret the result of one approach in…
Applied statistical problems often come with pre-specified groupings to predictors. It is natural to test for the presence of simultaneous group-wide signal for groups in isolation, or for multiple groups together. Classical tests for the…
Across the empirical sciences, few statistical procedures rival the popularity of the frequentist t-test. In contrast, the Bayesian versions of the t-test have languished in obscurity. In recent years, however, the theoretical and practical…
This paper presents a Bayesian framework for assessing the adequacy of a model without the necessity of explicitly enumerating a specific alternate model. A test statistic is developed for tracking the performance of the model across…
It is observed that for testing between simple hypotheses where the cost of Type I and Type II errors can be quantified, it is better to let the optimization choose the test size.
A number of applications require two-sample testing on ranked preference data. For instance, in crowdsourcing, there is a long-standing question of whether pairwise comparison data provided by people is distributed similar to…
Cluster analysis is a fundamental research issue in statistics and machine learning. In many modern clustering methods, we need to determine whether two subsets of samples come from the same cluster. Since these subsets are usually…
The so-called 'energy test' is a frequentist technique used in experimental particle physics to decide whether two samples are drawn from the same distribution. Its usage requires a good understanding of the distribution of the test…