Related papers: Variance estimation for Brier Score decomposition

When predicting future events, it is common to issue forecasts that are probabilistic, in the form of probability distributions over the range of possible outcomes. Such forecasts can be evaluated using proper scoring rules. Proper scoring…

Previously [Journal of Causal Inference, 10, 90-105 (2022)], we computed the variance of two estimators of causal effects for a v-structure of binary variables. Here we show that a linear combination of these estimators has lower variance…

In empirical research, when we have multiple estimators for the same parameter of interest, a central question arises: how do we combine unbiased but less precise estimators with biased but more precise ones to improve the inference? Under…

Uncertainty around multimodel ensemble forecasts of changes in future climate reduces the accuracy of those forecasts. For very uncertain forecasts this effect may mean that the forecasts should not be used. We investigate the use of the…

A composite likelihood is an inference function derived by multiplying a set of likelihood components. This approach provides a flexible framework for drawing inference when the likelihood function of a statistical model is computationally…

Combining measurements which have "theoretical uncertainties" is a delicate matter, due to an unclear statistical basis. We present an algorithm based on the notion that a theoretical uncertainty represents an estimate of bias.

Penalized logistic regression methods are frequently used to investigate the relationship between a binary outcome and a set of explanatory variables. The model performance can be assessed by measures such as the concordance statistic…

Bivariate partial information decompositions (PIDs) characterize how the information in a "message" random variable is decomposed between two "constituent" random variables in terms of unique, redundant and synergistic information…

Multi-step forecasting is often described through a simple rule of thumb: recursive strategies are said to have high bias and low variance, while direct strategies are said to have low bias and high variance. We revisit this belief by…

The relevance of determinacy coefficients as indicators for the validity of factor score predictors has regularly been emphasized. Previous simulation studies revealed biased determinacy coefficients for factor score predictors based on…

In healthcare applications, predictive uncertainty has been used to assess predictive accuracy. In this paper, we demonstrate that predictive uncertainty estimated by the current methods does not highly correlate with prediction error by…

The Bayesian Cram\'er-Rao bound (CRB) provides a lower bound on the mean square error of any Bayesian estimator under mild regularity conditions. It can be used to benchmark the performance of statistical estimators, and provides a…

This paper develops maximum score estimation of preference parameters in the binary choice model under uncertainty in which the decision rule is affected by conditional expectations. The preference parameters are estimated in two stages: we…

In the report the approach to estimation of quality of planned experiments is considered. This approach is based on the analysis of uncertainty, which will take place under the future hypotheses testing about the existence of a new…

Proper scoring rules evaluate the quality of probabilistic predictions, playing an essential role in the pursuit of accurate and well-calibrated models. Every proper score decomposes into two fundamental components -- proper calibration…

This paper considers the problem of estimating the variance of a sum of a triangular array of random vectors with heterogeneous means. When random vectors exhibit two-way cluster dependence or weak dependence, standard variance estimators…

Model calibration is essential for ensuring that the predictions of deep neural networks accurately reflect true probabilities in real-world classification tasks. However, deep networks often produce over-confident or under-confident…

Contemporary sample size calculations for external validation of risk prediction models require users to specify fixed values of assumed model performance metrics alongside target precision levels (e.g., 95% CI widths). However, due to the…

Probabilistic forecasts must sum to unity and cannot express ``I don't know.'' Possibility theory relaxes this constraint: a subnormal distribution explicitly measures how much of the plausibility budget remains unassigned, ignorance signal…

Suppose we are interested in the effect of variable $X$ on variable $Y$. If $X$ and $Y$ both influence, or are associated with variables that influence, a common outcome, called a collider, then conditioning on the collider (or on a…