English
Related papers

Related papers: Cross-Validated Loss-Based Covariance Matrix Estim…

200 papers

Data analysis in cosmology requires reliable covariance matrices. Covariance matrices derived from numerical simulations often require a very large number of realizations to be accurate. When a theoretical model for the covariance matrix…

Cosmology and Nongalactic Astrophysics · Physics 2022-12-21 Alessandra Fumagalli , Matteo Biagetti , Alexandro Saro , Emiliano Sefusatti , Anže Slosar , Pierluigi Monaco , Alfonso Veropalumbo

We propose and analyze a new estimator of the covariance matrix that admits strong theoretical guarantees under weak assumptions on the underlying distribution, such as existence of moments of only low order. While estimation of covariance…

Statistics Theory · Mathematics 2018-01-17 Stanislav Minsker , Xiaohan Wei

Cross-validation (CV) is often used to select the regularization parameter in high dimensional problems. However, when applied to the sparse modeling method Lasso, CV leads to models that are unstable in high-dimensions, and consequently…

Methodology · Statistics 2015-10-28 Chinghway Lim , Bin Yu

Inference and Estimation in Missing Information (MI) scenarios are important topics in Statistical Learning Theory and Machine Learning (ML). In ML literature, attempts have been made to enhance prediction through precise feature selection…

Machine Learning · Statistics 2017-07-26 Ahmadreza Moradipari , Sina Shahsavari , Ashkan Esmaeili , Farokh Marvasti

We derive high-dimensional Gaussian comparison results for the standard $V$-fold cross-validated risk estimates. Our results combine a recent stability-based argument for the low-dimensional central limit theorem of cross-validation with…

Statistics Theory · Mathematics 2023-11-15 Nicholas Kissel , Jing Lei

Covariance matrix estimation is an important problem in multivariate data analysis, both from theoretical as well as applied points of view. Many simple and popular covariance matrix estimators are known to be severely affected by model…

Methodology · Statistics 2025-11-21 Soumya Chakraborty , Ayanendranath Basu , Abhik Ghosh

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain…

Methodology · Statistics 2022-04-26 Xiaoyi Wang , Gongjun Xu , Shurong Zheng

This text is a survey on cross-validation. We define all classical cross-validation procedures, and we study their properties for two different goals: estimating the risk of a given estimator, and selecting the best estimator among a given…

Statistics Theory · Mathematics 2017-03-10 Sylvain Arlot

We consider the estimation of integrated covariance (ICV) matrices of high dimensional diffusion processes based on high frequency observations. We start by studying the most commonly used estimator, the realized covariance (RCV) matrix. We…

Methodology · Statistics 2015-03-17 Xinghua Zheng , Yingying Li

We propose a general framework for nonasymptotic covariance matrix estimation making use of concentration inequality-based confidence sets. We specify this framework for the estimation of large sparse covariance matrices through…

Methodology · Statistics 2020-12-17 Adam B Kashlak , Linglong Kong

Covariance estimation becomes challenging in the regime where the number p of variables outstrips the number n of samples available to construct the estimate. One way to circumvent this problem is to assume that the covariance matrix is…

Probability · Mathematics 2012-06-14 Richard Y. Chen , Alex Gittens , Joel A. Tropp

Estimation and hypothesis tests for the covariance matrix in high dimensions is a challenging problem as the traditional multivariate asymptotic theory is no longer valid. When the dimension is larger than or increasing with the sample…

Methodology · Statistics 2020-11-18 Deepak Nag Ayyala , Santu Ghosh , Daniel F. Linder

In this paper we study covariance estimation with missing data. We consider missing data mechanisms that can be independent of the data, or have a time varying dependency. Additionally, observed variables may have arbitrary (non uniform)…

Statistics Theory · Mathematics 2021-06-17 Eduardo Pavez , Antonio Ortega

Distributional approximations of (bi--) linear functions of sample variance-covariance matrices play a critical role to analyze vector time series, as they are needed for various purposes, especially to draw inference on the dependence…

Probability · Mathematics 2018-03-20 Ansgar Steland , Rainer von Sachs

This paper studies the high-dimensional mixed linear regression (MLR) where the output variable comes from one of the two linear regression models with an unknown mixing proportion and an unknown covariance structure of the random…

Methodology · Statistics 2020-11-10 Linjun Zhang , Rong Ma , T. Tony Cai , Hongzhe Li

In this paper, we present several estimators of the diagonal elements of the inverse of the covariance matrix, called precision matrix, of a sample of iid random vectors. The focus is on high dimensional vectors having a sparse precision…

Statistics Theory · Mathematics 2017-07-31 Samuel Balmand , Arnak S. Dalalyan

Unbiased and consistent variance estimators generally do not exist for design-based treatment effect estimators because experimenters never observe more than one potential outcome for any unit. The problem is exacerbated by interference and…

Methodology · Statistics 2024-07-04 Christopher Harshaw , Joel A. Middleton , Fredrik Sävje

This manuscript presents an approach to perform generalized linear regression with multiple high dimensional covariance matrices as the outcome. Model parameters are proposed to be estimated by maximizing a pseudo-likelihood. When the data…

Methodology · Statistics 2020-07-28 Yi Zhao , Brian S. Caffo , Xi Luo

We study the problem of treatment effect estimation in randomized experiments with high-dimensional covariate information, and show that essentially any risk-consistent regression adjustment can be used to obtain efficient estimates of the…

Methodology · Statistics 2022-06-08 Stefan Wager , Wenfei Du , Jonathan Taylor , Robert Tibshirani

We develop tools to do valid post-selective inference for a family of model selection procedures, including choosing a model via cross-validated Lasso. The tools apply universally when the following random vectors are jointly asymptotically…

Methodology · Statistics 2018-02-13 Jelena Markovic , Lucy Xia , Jonathan Taylor