Related papers: Accuracy, Repeatability, and Reproducibility of Fi…
When testing multiple hypothesis in a survey --e.g. many different source locations, template waveforms, and so on-- the final result consists in a set of confidence intervals, each one at a desired confidence level. But the probability…
We propose the use of a simple intuitive principle for measuring algorithmic classification bias: the significance of the differences in a classifier's error rates across the various demographics is inversely commensurate with the sample…
Count outcomes in longitudinal studies are frequent in clinical and engineering studies. In frequentist and Bayesian statistical analysis, methods such as Mixed linear models allow the variability or correlation within individuals to be…
The assumption of free will - the ability of an experimentalist to make random choices - is central to proving the indeterminism of quantum resources, the primary tool in quantum cryptography. Relaxing the assumption in a Bell test allows…
We compare and contrast the error probability and fidelity as measures of the quality of the receiver's measurement strategy for a quantum communications system. The error probability is a measure of the ability to retrieve {\it classical}…
Biomedical retrieval-augmented large language models (LLMs) often face evidence that is incomplete, misleading, or internally contradictory, yet evaluation usually emphasizes answer accuracy under helpful context rather than reliability…
The performance of a binary classifier is described by a confusion matrix with four entries: the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The Matthews Correlation Coefficient (MCC),…
In small sample studies with binary outcome data, use of a normal approximation for hypothesis testing can lead to substantial inflation of the type-I error-rate. Consequently, exact statistical methods are necessitated, and accordingly,…
To evaluate the calibration of a disease risk prediction tool, the quantity $E/O$, i.e., the ratio of the expected number of events to the observed number of events, is generally computed. However, because of censoring, or more precisely…
Plausibility is a formalization of exact tests for parametric models and generalizes procedures such as Fisher's exact test. The resulting tests are based on cumulative probabilities of the probability density function and evaluate…
Measurement error in multinomial data is a well-known and well-studied inferential problem that is encountered in many fields, including engineering, biomedical and omics research, ecology, finance, official statistics, and social sciences.…
There has been a rise in third-party cloud providers offering quantum hardware as a service to improve performance at lower cost. Although these providers provide flexibility to the users to choose from several qubit technologies, quantum…
There is a growing literature on reasoning by large language models (LLMs), but the discussion on the uncertainty in their responses is still lacking. Our aim is to assess the extent of confidence that LLMs have in their answers and how it…
The negative-binomial distribution is adopted for analyzing asbestos-fiber counts so as to account for both the sampling errors in capturing only a finite number of fibers as well as the inevitable human variation in identifying and…
In data mining, when binary prediction rules are used to predict a binary outcome, many performance measures are used in a vast array of literature for the purposes of evaluation and comparison. Some examples include classification…
Following recent shootings in the USA, a debate has erupted, one side favoring stricter gun control, the other promoting protection through more weapons. We provide a scientific foundation to inform this debate, based on mathematical,…
In applications such as end-to-end encrypted instant messaging, secure email, and device pairing, users need to compare key fingerprints to detect impersonation and adversary-in-the-middle attacks. Key fingerprints are usually computed as…
We derive the sample size formulae for comparing two negative binomial rates based on both the relative and absolute rate difference metrics in noninferiority and equivalence trials with unequal follow-up times, and establish an approximate…
Adversarial Training is proved to be an efficient method to defend against adversarial examples, being one of the few defenses that withstand strong attacks. However, traditional defense mechanisms assume a uniform attack over the examples…
Commonly used evaluation measures including Recall, Precision, F-Measure and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the…