English
Related papers

Related papers: A Python Library For Empirical Calibration

200 papers

Parametric empirical Bayes (EB) estimators have been widely used in variety of fields including small area estimation, disease mapping. Since EB estimator is constructed by plugging in the estimator of parameters in prior distributions, it…

Methodology · Statistics 2017-04-28 Shonosuke Sugasawa

Bayesian statistics has gained popularity in psychological research due to its intuitive uncertainty quantification and convenient information-updating rules. In many applications, however, prior distributions are introduced merely as…

Methodology · Statistics 2026-03-10 Yang Liu , Jonathan P. Williams , Jan Hannig

This paper proposes a new metric to measure the calibration error of probabilistic binary classifiers, called test-based calibration error (TCE). TCE incorporates a novel loss function based on a statistical test to examine the extent to…

Machine Learning · Statistics 2023-06-27 Takuo Matsubara , Niek Tax , Richard Mudd , Ido Guy

For many applications of probabilistic classifiers it is important that the predicted confidence vectors reflect true probabilities (one says that the classifier is calibrated). It has been shown that common models fail to satisfy this…

Machine Learning · Statistics 2022-10-10 Michael Panchenko , Anes Benmerzoug , Miguel de Benito Delgado

We study methods for simultaneous analysis of many noisy experiments in the presence of rich covariate information. The goal of the analyst is to optimally estimate the true effect underlying each experiment. Both the noisy experimental…

Methodology · Statistics 2020-01-14 Nikolaos Ignatiadis , Stefan Wager

Entity resolution (ER), comprising record linkage and de-duplication, is the process of merging noisy databases in the absence of unique identifiers to remove duplicate entities. One major challenge of analysis with linked data is…

Methodology · Statistics 2022-02-10 Andee Kaplan , Brenda Betancourt , Rebecca C. Steorts

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to 'mine' variables of interest…

Econometrics · Economics 2020-12-22 Mochen Yang , Edward McFowland , Gordon Burtch , Gediminas Adomavicius

Two-phase sampling designs are frequently employed in epidemiological studies and large-scale health surveys. In such designs, certain variables are exclusively collected within a second-phase random subsample of the initial first-phase…

Methodology · Statistics 2024-03-25 Lingxiao Wang

Data clustering is a fundamental problem with a wide range of applications. Standard methods, eg the $k$-means method, usually require solving a non-convex optimization problem. Recently, total variation based convex relaxation to the…

Optimization and Control · Mathematics 2018-08-29 Guodong Xu , Yu Xia , Hui Ji

Optimal decision making requires that classifiers produce uncertainty estimates consistent with their empirical accuracy. However, deep neural networks are often under- or over-confident in their predictions. Consequently, methods have been…

Regression calibration as developed by Rosner, Spiegelman and Willet is used to correct the bias in effect estimates due to measurement error in continuous exposures. The method involves two models: a measurement error model (MEM) relating…

Methodology · Statistics 2026-02-24 Wenze Tang , Donna Spiegelman , Xiaomei Liao , Molin Wang

Algorithmic fairness has received considerable attention due to the failures of various predictive AI systems that have been found to be unfairly biased against subgroups of the population. Many approaches have been proposed to mitigate…

Machine Learning · Computer Science 2025-03-14 Agathe Fernandes Machado , Suzie Grondin , Philipp Ratz , Arthur Charpentier , François Hu

The deployment of machine learning classifiers in high-stakes domains requires well-calibrated confidence scores for model predictions. In this paper we introduce the notion of variable-based calibration to characterize calibration…

Machine Learning · Computer Science 2023-04-07 Markelle Kelly , Padhraic Smyth

This work describes the TrueLearn Python library, which contains a family of online learning Bayesian models for building educational (or more generally, informational) recommendation systems. This family of models was designed following…

Information Retrieval · Computer Science 2023-09-22 Yuxiang Qiu , Karim Djemili , Denis Elezi , Aaneel Shalman , María Pérez-Ortiz , Sahan Bulathwela

Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not…

Machine Learning · Statistics 2019-11-01 Jayaraman J. Thiagarajan , Bindya Venkatesh , Deepta Rajan

A common goal in observational research is to estimate marginal causal effects in the presence of confounding variables. One solution to this problem is to use the covariate distribution to weight the outcomes such that the data appear…

Methodology · Statistics 2020-08-18 Kevin P. Josey , Elizabeth Juarez-Colunga , Fan Yang , Debashis Ghosh

A probabilistic model is said to be calibrated if its predicted probabilities match the corresponding empirical frequencies. Calibration is important for uncertainty quantification and decision making in safety-critical applications. While…

Machine Learning · Computer Science 2020-07-01 Anusri Pampari , Stefano Ermon

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce…

Methodology · Statistics 2020-10-22 Sean Talts , Michael Betancourt , Daniel Simpson , Aki Vehtari , Andrew Gelman

In observational causal inference, domain knowledge often leaves multiple covariate adjustments plausible, yet which sets satisfy ignorability is untestable. Different adjustment sets can yield conflicting estimates of the average treatment…

Methodology · Statistics 2026-03-23 Aditya Ghosh , Dominik Rothenhäusler

In the recent literature on machine learning and decision making, calibration has emerged as a desirable and widely-studied statistical property of the outputs of binary prediction models. However, the algorithmic aspects of measuring model…

Machine Learning · Computer Science 2024-06-24 Lunjia Hu , Arun Jambulapati , Kevin Tian , Chutong Yang