English
Related papers

Related papers: Do more observations bring more information in rar…

200 papers

Post-selection inference has recently been proposed as a way of quantifying uncertainty about detected changepoints. The idea is to run a changepoint detection algorithm, and then re-use the same data to perform a test for a change near…

Methodology · Statistics 2026-05-11 Rachel Carrington , Paul Fearnhead

Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with…

Data Analysis, Statistics and Probability · Physics 2021-03-15 Grant M. Rotskoff , Andrew R. Mitchell , Eric Vanden-Eijnden

The large-scale multiple testing inherent to high throughput biological data necessitates very high statistical stringency and thus true effects in data are difficult to detect unless they have high effect sizes. One solution to this…

Methodology · Statistics 2017-12-21 Mohamad S. Hasan

Data-driven most powerful tests are statistical hypothesis decision-making tools that deliver the greatest power against a fixed null hypothesis among all corresponding data-based tests of a given size. When the underlying data…

Statistics Theory · Mathematics 2023-03-15 Albert Vexler , Alan D. Hutson

The probability of rare and extreme events is an important quantity for design purposes. However, computing the probability of rare events can be expensive because only a few events, if any, can be observed. To this end, it is necessary to…

Computational Physics · Physics 2020-01-08 Malik Hassanaly , Venkat Raman

This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical…

Computation · Statistics 2009-10-13 Michel Broniatowski , Ya'Acov Ritov

We consider the problem of choosing design parameters to minimize the probability of an undesired rare event that is described through the average of $n$ iid random variables. Since the probability of interest for near optimal design…

Optimization and Control · Mathematics 2019-02-22 Amarjit Budhiraja , Shu Lu , Yang Yu , Quoc Tran-Dinh

Experiments often yield non-identically distributed data for statistical analysis. Tests of hypothesis under such set-ups are generally performed using the likelihood ratio test, which is non-robust with respect to outliers and model…

Statistics Theory · Mathematics 2017-07-25 Abhik Ghosh , Ayanendranath Basu

Rare events are events that are expected to occur infrequently, or more technically, those that have low probabilities (say, order of $10^{-3}$ or less) of occurring according to a probability model. In the context of uncertainty…

Computation · Statistics 2015-08-21 James L. Beck , Konstantin M. Zuev

In modern scientific research, small-scale studies with limited participants are increasingly common. However, interpreting individual outcomes can be challenging, making it standard practice to combine data across studies using random…

Statistics Theory · Mathematics 2025-11-04 Lucas Kania , Larry Wasserman , Sivaraman Balakrishnan

We study conditions under which the addition of variables to a regression equation can turn a previously statistically insignificant result into a significant one. Specifically, we characterize the minimum strength of association required…

Statistics Theory · Mathematics 2025-09-24 Danielle Tsao , Ronan Perry , Carlos Cinelli

A vast array of transformative technologies developed over the past decade has enabled measurement and perturbation at ever increasing scale, yet our understanding of many systems remains limited by experimental capacity. Overcoming this…

Quantitative Methods · Quantitative Biology 2020-12-25 Brian Cleary , Aviv Regev

Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in small…

Methodology · Statistics 2023-06-13 Adam C. Sales , Ethan B. Prihar , Johann A. Gagnon-Bartsch , Neil T. Heffernan

We investigate the predictability of extreme events in time series. The focus of this work is to understand under which circumstances large events are better predictable than smaller events. Therefore we use a simple prediction algorithm…

Data Analysis, Statistics and Probability · Physics 2008-01-30 S. Hallerberg , H. Kantz

Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…

Data Analysis, Statistics and Probability · Physics 2019-07-24 Damián G. Hernández , Inés Samengo

Compositional data (i.e., data comprising random variables that sum up to a constant) arises in many applications including microbiome studies, chemical ecology, political science, and experimental designs. Yet when compositional data serve…

Methodology · Statistics 2025-01-03 Ritwik Bhaduri , Siyuan Ma , Lucas Janson

Estimating the causal effects of an intervention from high-dimensional observational data is difficult due to the presence of confounding. The task is often complicated by the fact that we may have a systematic missingness in our data at…

Machine Learning · Statistics 2020-03-02 Sonali Parbhoo , Mario Wieser , Aleksander Wieczorek , Volker Roth

Extracting maximal information from experimental data requires access to the likelihood function, which however is never directly available for complex experiments like those performed at high energy colliders. Theoretical predictions are…

High Energy Physics - Phenomenology · Physics 2023-08-11 Siyu Chen , Alfredo Glioti , Giuliano Panico , Andrea Wulzer

In multigroup data settings with small within-group sample sizes, standard $F$-tests of group-specific linear hypotheses can have low power, particularly if the within-group sample sizes are not large relative to the number of explanatory…

Methodology · Statistics 2022-03-25 Andrew McCormack , Peter Hoff

Genome-wide association analysis has generated much discussion about how to preserve power to detect signals despite the detrimental effect of multiple testing on power. We develop a weighted multiple testing procedure that facilitates the…

Statistics Theory · Mathematics 2007-06-13 Kathryn Roeder , Bernie Devlin , Larry Wasserman