Related papers: Self-Calibrating the Look-Elsewhere Effect: Fast E…
The search for new significant peaks over a energy spectrum often involves a statistical multiple hypothesis testing problem. Separate tests of hypothesis are conducted at different locations producing an ensemble of local p-values, the…
When searching over a large parameter space for anomalies such as events, peaks, objects, or particles, there is a large probability that spurious signals with seemingly high significance will be found. This is known as the look-elsewhere…
Machine learning-based anomaly detection methods are able to search high-dimensional spaces for hints of new physics with much less theory bias than traditional searches. However, by searching in many directions all at once, the statistical…
Periodograms are common tools used to search for periodic signals in unevenly spaced time series. The significance of periodogram peaks is often assessed using false alarm probability (FAP), which in most studies assumes uncorrelated noise…
Peak inference concerns the use of local maxima ("peaks") of a noisy random field to detect and localize regions where underlying signal is present. We propose a peak inference method that first subjects observed peaks to a significance…
When searching for a new resonance somewhere in a possible mass range, the significance of observing a local excess of events must take into account the probability of observing such an excess anywhere in the range. This is the so called…
We discuss the use of Gaussian random fields to estimate the look-elsewhere effect correction. We show that Gaussian random fields can be used to model the null-hypothesis significance maps from a large set of statistical problems commonly…
Context. In exoplanet searches with radial velocity data, the most common statistical significance metrics are the Bayes factor and the false alarm probability (FAP). Both have proved useful, but do not directly address whether an exoplanet…
In high Energy Physics, it is common to look for a localized deviation in data with respect to a given reference. For this task, the well known BumpHunter algorithm allows for a model-independent deviation search with the advantage of…
The experimental issue of the search for new particles of unknown mass poses the challenge of exploring a wide interval to look for the usual signatures represented by excess of events above the background. A side effect of such a broad…
Strong gravitational lensing of gravitational waves (GWs) has been forecasted to become detectable in the upcoming observing runs. However, definitively distinguishing pairs of lensed sources from random associations is a challenging…
We consider the common setting where one observes probability estimates for a large number of events, such as default risks for numerous bonds. Unfortunately, even with unbiased estimates, selecting events corresponding to the most extreme…
We study the properties of several likelihood-based statistics commonly used in testing for the presence of a known signal under a mixture model with known background, but unknown signal fraction. Under the null hypothesis of no signal, all…
Several construction methods for rational approximations to functions of one real variable are described in the present paper; the computational results that characterize the comparative accuracy of these methods are presented; an effect of…
Probabilistic classifiers output confidence scores along with their predictions, and these confidence scores should be calibrated, i.e., they should reflect the reliability of the prediction. Confidence scores that minimize standard metrics…
Anomaly detection is the task of identifying examples that do not behave as expected. Because anomalies are rare and unexpected events, collecting real anomalous examples is often challenging in several applications. In addition, learning…
Response calibration is the process of inferring how much the measured data depend on the signal one is interested in. It is essential for any quantitative signal estimation on the basis of the data. Here, we investigate self-calibration…
Probability predictions from binary regressions or machine learning methods ought to be calibrated: If an event is predicted to occur with probability $x$, it should materialize with approximately that frequency, which means that the…
Quantifying the usefulness of individual features in random forests learning can greatly enhance its interpretability. Existing studies have shown that some popularly used feature importance measures for random forests suffer from the bias…
A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data "peeking" -- the practice of reporting only progressively extreme values of the test statistic as more data samples are…