English
Related papers

Related papers: Covariance Matrix Estimation for High-Dimensional …

200 papers

It has been proposed that complex populations, such as those that arise in genomics studies, may exhibit dependencies among observations as well as among variables. This gives rise to the challenging problem of analyzing unreplicated…

Machine Learning · Statistics 2018-06-08 Michael Hornstein , Roger Fan , Kerby Shedden , Shuheng Zhou

Ideally, all analyses of normally distributed data should include the full covariance information between all data points. In practice, the full covariance matrix between all data points is not always available. Either because a result was…

Methodology · Statistics 2026-02-23 Lukas Koch

In dealing with high-dimensional data, factor models are often used for reducing dimensions and extracting relevant information. The spectrum of covariance matrices from power data exhibits two aspects: 1) bulk, which arises from random…

Applications · Statistics 2019-10-22 Xin Shi , Robert Qiu

Because of the advance in technologies, modern statistical studies often encounter linear models with the number of explanatory variables much larger than the sample size. Estimation and variable selection in these high-dimensional problems…

Statistics Theory · Mathematics 2012-06-06 Jun Shao , Xinwei Deng

The problem of estimating a normal covariance matrix is considered from a decision-theoretic point of view, where the dimension of the covariance matrix is larger than the sample size. This paper addresses not only the nonsingular case but…

Statistics Theory · Mathematics 2015-06-03 Hisayuki Tsukuma

Estimating high-dimensional covariance matrices is a key task across many fields. This paper explores the theoretical limits of distributed covariance estimation in a feature-split setting, where communication between agents is constrained.…

Machine Learning · Statistics 2025-07-24 Mohammad Reza Rahmani , Mohammad Hossein Yassaee , Mohammad Reza Aref

Fitting high-dimensional data involves a delicate tradeoff between faithful representation and the use of sparse models. Too often, sparsity assumptions on the fitted model are too restrictive to provide a faithful representation of the…

Machine Learning · Statistics 2013-12-17 Majid Janzamin , Animashree Anandkumar

As a classical problem, covariance estimation has drawn much attention from the statistical community for decades. Much work has been done under the frequentist and the Bayesian frameworks. Aiming to quantify the uncertainty of the…

Methodology · Statistics 2017-08-17 W. Jenny Shi , Jan Hannig , Randy C. S. Lai , Thomas C. M. Lee

We present an estimator of the covariance matrix $\Sigma$ of random $d$-dimensional vector from an i.i.d. sample of size $n$. Our sole assumption is that this vector satisfies a bounded $L^p-L^2$ moment assumption over its one-dimensional…

Statistics Theory · Mathematics 2024-03-27 Roberto I. Oliveira , Zoraida F. Rico

The non-parametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are…

Statistics Theory · Mathematics 2022-01-19 Tomas Masak , Soham Sarkar , Victor M. Panaretos

Statistical analysis of high-dimensional functional times series arises in various applications. Under this scenario, in addition to the intrinsic infinite-dimensionality of functional data, the number of functional variables can grow with…

Statistics Theory · Mathematics 2022-01-14 Qin Fang , Shaojun Guo , Xinghao Qiao

This paper develops an inferential theory for high-dimensional matrix-variate factor models with missing observations. We propose an easy-to-use all-purpose method that involves two straightforward steps. First, we perform principal…

Methodology · Statistics 2025-03-26 Yongxia Zhang , Jinwen Liang , Liwen Xu , Keming Yu , Maozai Tian

Compositional data arise in many areas of research in the natural and biomedical sciences. One prominent example is in the study of the human gut microbiome, where one can measure the relative abundance of many distinct microorganisms in a…

Methodology · Statistics 2024-04-26 Aaron J. Molstad , Karl Oskar Ekvall , Piotr M. Suder

In this paper we consider estimation of sparse covariance matrices and propose a thresholding procedure which is adaptive to the variability of individual entries. The estimators are fully data driven and enjoy excellent performance both…

Methodology · Statistics 2011-02-14 Tony Cai , Weidong Liu

We consider testing the equality of two high-dimensional covariance matrices by carrying out a multi-level thresholding procedure, which is designed to detect sparse and faint differences between the covariances. A novel U-statistic…

Statistics Theory · Mathematics 2019-10-30 Song Xi Chen , Bin Guo , Yumou Qiu

Matrix factor model is drawing growing attention for simultaneous two-way dimension reduction of well-structured matrix-valued observations. This paper focuses on robust statistical inference for matrix factor model in the ``diverging…

Methodology · Statistics 2023-06-07 Yong He , Xin-Bing Kong , Dong Liu , Ran Zhao

Statistical learning evolves quickly with more and more sophisticated models proposed to incorporate the complicated data structure from modern scientific and business problems. Varying index coefficient models extend varying coefficient…

Statistics Theory · Mathematics 2019-03-05 Li Jialiang , Lv Jing

Variable selection plays a fundamental role in high-dimensional data analysis. Various methods have been developed for variable selection in recent years. Well-known examples are forward stepwise regression (FSR) and least angle regression…

Methodology · Statistics 2018-02-01 Siliang Gong , Kai Zhang , Yufeng Liu

Quantile regression has been successfully used to study heterogeneous and heavy-tailed data. Varying-coefficient models are frequently used to capture changes in the effect of input variables on the response as a function of an index or…

Methodology · Statistics 2021-10-18 Ran Dai , Mladen Kolar

It is of importance to develop statistical techniques to analyze high-dimensional data in the presence of both complex dependence and possible outliers in real-world applications such as imaging data analyses. We propose a new robust…

Methodology · Statistics 2021-10-01 Bingyuan Liu , Qi Zhang , Lingzhou Xue , Peter X. K. Song , Jian Kang