Related papers: Comparing Confidence Intervals for a Binomial Prop…
A new method is proposed for the correction of confidence intervals when the original interval does not have the correct nominal coverage probabilities in the frequentist sense. The proposed method is general and does not require any…
For the quantification of QoE, subjects often provide individual rating scores on certain rating scales which are then aggregated into Mean Opinion Scores (MOS). From the observed sample data, the expected value is to be estimated. While…
Introductory texts on statistics typically only cover the classical "two sigma" confidence interval for the mean value and do not describe methods to obtain confidence intervals for other estimators. The present technical report fills this…
We consider the classic problem of interval estimation of a proportion $p$ based on binomial sampling. The "exact" Clopper-Pearson confidence interval for $p$ is known to be unnecessarily conservative. We propose coverage-adjustments of the…
This short study presents an opportunistic approach to a (more) reliable validation method for prediction uncertainty average calibration. Considering that variance-based calibration metrics (ZMS, NLL, RCE...) are quite sensitive to the…
Any supervised machine learning analysis is required to provide an estimate of the out-of-sample predictive performance. However, it is imperative to also provide a quantification of the uncertainty of this performance in the form of a…
In this paper, we derive an explicit formula for constructing the confidence interval of binomial parameter with guaranteed coverage probability. The formula overcomes the limitation of normal approximation which is asymptotic in nature and…
A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The confidence interval is consistent with an exact unconditional test that preserves the type-I error, and has improved power,…
We present a novel and easy-to-use method for calibrating error-rate based confidence intervals to evidence-based support intervals. Support intervals are obtained from inverting Bayes factors based on a parameter estimate and its standard…
Forecasts for uncertain future events should be probabilistic. Probabilistic forecasts are commonly issued as prediction intervals, which provide a measure of uncertainty in the unknown outcome whilst being easier to understand and…
We consider the problem of interval estimation of the odds ratio. An asymptotic confidence interval is widely applied in medical research. Unfortunately that confidence interval has a poor coverage probability: it is significantly smaller…
In Natural Language Processing (NLP), binary classification algorithms are often evaluated using the F1 score. Because the sample F1 score is an estimate of the population F1 score, it is not sufficient to report the sample F1 score without…
This essay looks at decision-making with interval-valued probability measures. Existing decision methods have either supplemented expected utility methods with additional criteria of optimality, or have attempted to supplement the…
This paper concerns the construction of confidence intervals in standard seroprevalence surveys. In particular, we discuss methods for constructing confidence intervals for the proportion of individuals in a population infected with a…
Confidence interval (CI) methods for stratified bilateral studies use intraclass correlation to avoid misleading results. In this article, we propose four CI methods (sample-size weighted global MLE-based Wald-type CI, complete MLE-based…
Estimating the probability of the binomial distribution is a basic problem, which appears in almost all introductory statistics courses and is performed frequently in various studies. In some cases, the parameter of interest is a difference…
Positive and negative likelihood ratios are parameters which are used to assess and compare the effectiveness of binary diagnostic tests. Both parameters only depend on the sensitivity and specificity of the diagnostic test and are…
The construction of the Bayesian credible (confidence) interval for a Poisson observable including both the signal and background with and without systematic uncertainties is presented. Introducing the conditional probability satisfying the…
We connect the power of Confidence Intervals in different Frequentist methods to their reliability. We show that in the case of a bounded parameter a biased method which near the boundary has large power in testing the parameter against…
We compute bias, variance, and approximate confidence intervals for the efficiency of a random selection process under various special conditions that occur in practical data analysis. We consider the following cases: a) the number of…