Related papers: Signal significance incorporating systematic uncer…
An exciting recent development is the uptake of deep neural networks in many scientific fields, where the main objective is outcome prediction with the black-box nature. Significance testing is promising to address the black-box issue and…
The estimated accuracy of a classifier is a random quantity with variability. A common practice in supervised machine learning, is thus to test if the estimated accuracy is significantly better than chance level. This method of signal…
The rapid advancement and widespread adoption of machine learning-driven technologies have underscored the practical and ethical need for creating interpretable artificial intelligence systems. Feature importance, a method that assigns…
Robustness of linear systems with constant coefficients is considered. There exist methods and tools for analyzing the stability of systems with random or deterministic uncertainties. At the same time, there are no approaches for the…
We analyze safety problems of complex systems using the methods of mathematical statistics for testing the output variables of a code simulating the operation of the system under consideration when the input variables are uncertain. We have…
We describe a method for computing the biases that systematic signals introduce in parameter estimation using a simple extension of the Fisher matrix formalism. This allows us to calculate the offset of the best fit parameters relative to…
We discuss a new method for setting limits on small signals in the presence of background noise. The method is based on a combination of a two dimensional confidence region and the large sample approximation to the likelihood ratio test…
Accurately estimating the proportion of true signals among a large number of variables is crucial for enhancing the precision and reliability of scientific research. Traditional signal proportion estimators often assume independence among…
In this paper we discuss several methods of significance calculation and point out the limits of their applicability. We then introduce a self consistent scheme for source detection and discuss some of its properties. The method allows…
Research often necessitates of samples, yet obtaining large enough samples is not always possible. When it is, the researcher may use one of two methods for deciding upon the required sample size: rules-of-thumb, quick yet uncertain, and…
Uncertainty quantification is a key part of astronomy and physics; scientific researchers attempt to model both statistical and systematic uncertainties in their data as best as possible, often using a Bayesian framework. Decisions might…
The rapid rise of large language models (LLMs) is reshaping the landscape of automatic assessment in education. While these systems demonstrate substantial advantages in adaptability to diverse question types and flexibility in output…
With the rise of network science old topics in ecology and economics are resurfacing. One such topic is structural stability (often referred to as qualitative stability or sign stability). A system is deemed structurally stable if the…
Background treatment is crucial to extract physics from precision experiments. In this paper, we introduce a novel method to assign each event a signal probability. This could then be used to weight the event's contribution to the…
Conformal prediction, which makes no distributional assumptions about the data, has emerged as a powerful and reliable approach to uncertainty quantification in practical applications. The nonconformity measure used in conformal prediction…
Experimental datasets are growing rapidly in size, scope, and detail, but the value of these datasets is limited by unwanted measurement noise. It is therefore tempting to apply analysis techniques that attempt to reduce noise and enhance…
Assessing image quality is crucial in image processing tasks such as compression, super-resolution, and denoising. While subjective assessments involving human evaluators provide the most accurate quality scores, they are impractical for…
Robustness checks are routine in empirical work, but there is no standard statistical procedure to formally measure what one can learn from them. I propose a "robustness radius" measure to quantify the amount by which the robustness checks…
This paper introduces a statistical method to decide whether two blocks in a pair of of images match reliably. The method ensures that the selected block matches are unlikely to have occurred "just by chance." The new approach is based on…
In the domain of physics experiments, data fitting is a pivotal technique for extracting insights from both experimental and simulated datasets. This article presents an approximation method designed to estimate the systematic errors…