Related papers: Convincing Evidence
This paper discusses different needs and approaches to establishing ``causation'' that are relevant in legal cases involving statistical input based on epidemiological (or more generally observational or population-based) information. We…
Matching for causal inference is a well-studied problem, but standard methods fail when the units to match are text documents: the high-dimensional and rich nature of the data renders exact matching infeasible, causes propensity scores to…
A decision procedure implemented over a computational trust mechanism aims to allow for decisions to be made regarding whether some entity or information should be trusted. As recognised in the literature, trust is contextual, and we…
Reliability is an essential measure of how closely observed scores represent latent scores (reflecting constructs), assuming some latent variable measurement model. We present a general theoretical framework of reliability, placing emphasis…
A fundamental problem in the practice and teaching of data science is how to evaluate the quality of a given data analysis, which is different than the evaluation of the science or question underlying the data analysis. Previously, we…
Building a safety case is a common approach to make expert judgement explicit about safety of a system. The issue of confidence in such argumentation is still an open research field. Providing quantitative estimation of confidence is an…
We study how to perform tests on samples of pairs of observations and predictions in order to assess whether or not the predictions are prudent. Prudence requires that that the mean of the difference of the observation-prediction pairs can…
All scientific interpretations of statistical outputs depend on background (auxiliary) assumptions that are rarely delineated or explicitly interrogated. These include not only the usual modeling assumptions, but also deeper assumptions…
Prediction credibility measures, in the form of confidence intervals or probability distributions, are fundamental in statistics and machine learning to characterize model robustness, detect out-of-distribution samples (outliers), and…
The ability to make decisions based on data, with its inherent uncertainties and variability, is a complex and vital skill in the modern world. The need for such quantitative critical thinking occurs in many different contexts, and while it…
In most prediction and estimation situations, scientists consider various statistical models for the same problem, and naturally want to select amongst the best. Hansen et al. (2011) provide a powerful solution to this problem by the…
A common approach to aggregate classification estimates in an ensemble of decision trees is to either use voting or to average the probabilities for each class. The latter takes uncertainty into account, but not the reliability of the…
Nearly all statistical analyses that inform policy-making are based on imperfect data. As examples, the data may suffer from measurement errors, missing values, sample selection bias, or record linkage errors. Analysts have to decide how to…
This article is geared towards theorists interested in estimating parameters of their theoretical models, and computing their own limits using available experimental data and elementary Mathematica code. The examples given can be useful…
The causal effect of an intervention (treatment/exposure) on an outcome can be estimated by: i) specifying knowledge about the data-generating process; ii) assessing under what assumptions a target quantity, such as for example a causal…
Researchers in psychology characterize decision-making as a process of eliminating options. While statistical modelling typically focuses on the eventual choice, we analyze consideration sets describing, for each survey participant, all…
Traditional reliability analysis has been using time to event data, degradation data, and recurrent event data, while the associated covariates tend to be simple and constant over time. Over the past years, we have witnessed the rapid…
It can be insightful to extend qualitative studies with a secondary quantitative analysis (where the former suggests insightful questions that the latter can answer). Documenting developer beliefs should be the start, not the end, of…
A key feature of human theory-of-mind is the ability to attribute beliefs to other agents as mentalistic explanations for their behavior. But given the wide variety of beliefs that agents may hold about the world and the rich language we…
Causality is a subject of philosophical debate and a central scientific issue with a long history. In the statistical domain, the study of cause and effect based on the notion of `fairness' in comparisons dates back several hundred years,…