English
Related papers

Related papers: Foundational principles for large scale inference:…

200 papers

We place ourselves in the setting of high-dimensional statistical inference where the number of variables $p$ in a dataset of interest is of the same order of magnitude as the number of observations $n$. We consider the spectrum of certain…

Statistics Theory · Mathematics 2010-01-05 Noureddine El Karoui

Statistical analysis of high-dimensional functional times series arises in various applications. Under this scenario, in addition to the intrinsic infinite-dimensionality of functional data, the number of functional variables can grow with…

Statistics Theory · Mathematics 2022-01-14 Qin Fang , Shaojun Guo , Xinghao Qiao

Standard penalized methods of variable selection and parameter estimation rely on the magnitude of coefficient estimates to decide which variables to include in the final model. However, coefficient estimates are unreliable when the design…

Methodology · Statistics 2018-02-13 Jonathan P Williams , Jan Hannig

In many settings, robust data analysis involves computational methods for uncertainty quantification and statistical inference. To design frequentist studies that leverage robust analysis methods, suitable sample sizes to achieve desired…

Methodology · Statistics 2025-12-19 Luke Hagar , Andrew J. Martin

The aim of this paper is to provide a resampling technique that allows us to make inference on superpopulation parameters in finite population setting. Under complex sampling designs, it is often difficult to obtain explicit results about…

Methodology · Statistics 2018-09-24 Pier Luigi Conti , Alberto Di Iorio

Large-scale empirical data, the sample size and the dimension are high, often exhibit various characteristics. For example, the noise term follows unknown distributions or the model is very sparse that the number of critical variables is…

Statistics Theory · Mathematics 2018-06-18 Yuehan Yang , Hu Yang

The purpose of this paper is to propose methodologies for statistical inference of low-dimensional parameters with high-dimensional data. We focus on constructing confidence intervals for individual coefficients and linear combinations of…

Methodology · Statistics 2012-11-05 Cun-Hui Zhang , Stephanie S. Zhang

An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. For example, the maximum likelihood estimator has a bias that can result in a significant inferential loss. This problem is…

Statistics Theory · Mathematics 2019-11-04 Stéphane Guerrier , Mucyo Karemera , Samuel Orso , Maria-Pia Victoria-Feser

Linear models are foundational tools in statistics and ubiquitous across the applied sciences. However, conventional statistical inference -- such as $t$-tests and $F$-tests -- are only valid at fixed sample sizes, making them unsuitable…

Methodology · Statistics 2025-07-08 Michael Lindon , Dae Woong Ham , Martin Tingley , Iavor Bojinov

A fundamental aspect of biological information processing is the ubiquity of sequence-function relationships -- functions that map the sequence of DNA, RNA, or protein to a biochemically relevant activity. Most sequence-function…

Quantitative Methods · Quantitative Biology 2016-03-23 Gurinder S. Atwal , Justin B. Kinney

With the advent of big data applications and the increasing amount of data being produced in these applications, the importance of efficient methods for big data analysis has become highly evident. However, the success of any such method…

Computers and Society · Computer Science 2019-11-05 Mostafa Mirzaie , Behshid Behkamal , Samad Paydar

The last decade witnessed a rise in the importance of supervised learning applications involving {\em big data} and {\em big models}. Big data refers to situations where the amounts of training data available and needed causes difficulties…

Optimization and Control · Mathematics 2018-11-01 Konstantin Mishchenko , Peter Richtárik

In the era of big data, analysts usually explore various statistical models or machine learning methods for observed data in order to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are…

Machine Learning · Statistics 2018-10-24 Jie Ding , Vahid Tarokh , Yuhong Yang

In processing large quantities of data, a fundamental problem is to obtain a summary which supports approximate query answering. Random sampling yields flexible summaries which naturally support subset-sum queries with unbiased estimators…

Data Structures and Algorithms · Computer Science 2011-02-28 Edith Cohen , Graham Cormode , Nick Duffield

In this paper, we consider procedures for testing hypotheses on the dimension of the linear span generated by a growing number of $p\times p$ covariance matrices from independent $q$ populations. Under a proper limiting scheme where all the…

Statistics Theory · Mathematics 2026-02-16 Tianxing Mei , Chen Wang , Jianfeng Yao

In scientific inference problems, the underlying statistical modeling assumptions have a crucial impact on the end results. There exist, however, only a few automatic means for validating these fundamental modelling assumptions. The…

Methodology · Statistics 2019-05-21 Andreas Svensson , Dave Zachariah , Petre Stoica , Thomas B. Schön

A vast array of transformative technologies developed over the past decade has enabled measurement and perturbation at ever increasing scale, yet our understanding of many systems remains limited by experimental capacity. Overcoming this…

Quantitative Methods · Quantitative Biology 2020-12-25 Brian Cleary , Aviv Regev

Generative models are invaluable in many fields of science because of their ability to capture high-dimensional and complicated distributions, such as photo-realistic images, protein structures, and connectomes. How do we evaluate the…

For high-dimensional inference problems, statisticians have a number of competing interests. On the one hand, procedures should provide accurate estimation, reliable structure learning, and valid uncertainty quantification. On the other…

Statistics Theory · Mathematics 2021-01-11 Ryan Martin

Effective utilization of time series data is often constrained by the scarcity of data quantity that reflects complex dynamics, especially under the condition of distributional shifts. Existing datasets may not encompass the full range of…

Computational Engineering, Finance, and Science · Computer Science 2024-06-11 Haibei Zhu , Yousef El-Laham , Elizabeth Fons , Svitlana Vyetrenko