Related papers: On likelihood ratio tests
This short text tried to establish a big picture of what evidential statistics is about and how an ideal inference method should behave. Moreover, by examining shortcomings of some of the currently used methods for measuring evidence and…
The empirical likelihood is a powerful nonparametric tool, that emulates its parametric counterpart -- the parametric likelihood -- preserving many of its large-sample properties. This article tackles the problem of assessing the…
When analyzing incomplete data, is it better to use multiple imputation (MI) or full information maximum likelihood (ML)? In large samples ML is clearly better, but in small samples ML's usefulness has been limited because ML commonly uses…
Large Language Models (LLMs) are increasingly used to evaluate information retrieval (IR) systems, generating relevance judgments traditionally made by human assessors. Recent empirical studies suggest that LLM-based evaluations often align…
We consider the problem of interval estimation of the odds ratio. An asymptotic confidence interval is widely applied in medical research. Unfortunately that confidence interval has a poor coverage probability: it is significantly smaller…
An agent often has a number of hypotheses, and must choose among them based on observations, or outcomes of experiments. Each of these observations can be viewed as providing evidence for or against various hypotheses. All the attempts to…
We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity against a composite one-sided parametric alternative that this is a…
The higher criticism of a family of tests starts with the individual uncorrected p-values of each test. It then requires a procedure for deciding whether the collection of p-values indicates the presence of a real effect and if possible…
In this paper, we consider one aspect of the problem of applying decision theory to the design of agents that learn how to make decisions under uncertainty. This aspect concerns how an agent can estimate probabilities for the possible…
Despite extensive theoretical research on proportionality in approval-based multiwinner voting, its impact on which committees and candidates can be selected in practice remains poorly understood. We address this gap by (i) analyzing the…
We propose an empirical likelihood test that is able to test the goodness of fit of a class of parametric and semi-parametric multiresponse regression models. The class includes as special cases fully parametric models; semi-parametric…
Unbinned maximum likelihood is a common procedure for parameter estimation. After parameters have been estimated, it is crucial to know whether the fit model adequately describes the experimental data. Univariate Goodness of Fit procedures…
We consider the problem of hypothesis testing in the situation where the first hypothesis is simple and the second one is local one-sided composite. We describe the choice of the thresholds and the power functions of different tests when…
In contrast to its common definition and calculation, interpretation of p-values diverges among statisticians. Since p-value is the basis of various methodologies, this divergence has led to a variety of test methodologies and evaluations…
We conduct an incentivized experiment on a nationally representative US sample \\ (N=708) to test whether people prefer to avoid ambiguity even when it means choosing dominated options. In contrast to the literature, we find that 55\% of…
Classifiers are often tested on relatively small data sets, which should lead to uncertain performance metrics. Nevertheless, these metrics are usually taken at face value. We present an approach to quantify the uncertainty of…
Quantiles can represent key operational and business metrics, but the computational challenges associated with inference has hampered their adoption in online experimentation. One-sample confidence intervals are trivial to construct;…
Although the halting problem is undecidable, imperfect testers that fail on some instances are possible. Such instances are called hard for the tester. One variant of imperfect testers replies "I don't know" on hard instances, another…
The development of science and technology has progressively demonstrated the ability of humankind to understand and manipulate the physical world, and it has also shown some fundamental limitations to predictability of physical events. This…
A group of approaches for calculating forensic likelihood ratios first calculates scores which quantify the degree of difference or the degree of similarity between pairs of samples, then converts those scores to likelihood ratios. In order…