English
Related papers

Related papers: Non-linear shrinkage estimation of large-scale str…

200 papers

Estimation of covariance matrices or their inverses plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. In this paper we present an…

Methodology · Statistics 2014-08-06 Eric C. Chi , Kenneth Lange

Data analysis from upcoming large galaxy redshift surveys, such as Euclid and DESI will significantly improve constraints on cosmological parameters. To optimally extract the information from these galaxy surveys, it is important to control…

Cosmology and Nongalactic Astrophysics · Physics 2025-01-22 S. Gouyou Beauchamps , P. Baratta , S. Escoffier , W. Gillard , J. Bel , J. Bautista , C. Carbone

Many popular statistical models, such as factor and random effects models, give arise a certain type of covariance structures that is a summation of low rank and sparse matrices. This paper introduces a penalized approximation framework to…

Methodology · Statistics 2015-03-19 Xi Luo

Third-order weak lensing statistics are a promising tool for cosmological analyses since they extract cosmological information in the non-Gaussianity of the cosmic large-scale structure. However, such analyses require precise and accurate…

Cosmology and Nongalactic Astrophysics · Physics 2023-04-26 Laila Linke , Sven Heydenreich , Pierre A. Burger , Peter Schneider

In this paper we derive the optimal linear shrinkage estimator for the high-dimensional mean vector using random matrix theory. The results are obtained under the assumption that both the dimension $p$ and the sample size $n$ tend to…

Statistics Theory · Mathematics 2018-07-17 Taras Bodnar , Ostap Okhrin , Nestor Parolya

Shrinkage can effectively improve the condition number and accuracy of covariance matrix estimation, especially for low-sample-support applications with the number of training samples smaller than the dimensionality. This paper investigates…

Information Theory · Computer Science 2018-10-22 Jun Tong , Rui Hu , Jiangtao Xi , Zhitao Xiao , Qinghua Guo , Yanguang Yu

This paper introduces a neural network-based nonlinear shrinkage estimator of covariance matrices for the purpose of minimum variance portfolio optimization. It is a hybrid approach that integrates statistical estimation with machine…

Machine Learning · Computer Science 2026-01-23 Liusha Yang , Siqi Zhao , Shuqi Chai

This paper introduces a subspace method for the estimation of an array covariance matrix. It is shown that when the received signals are uncorrelated, the true array covariance matrices lie in a specific subspace whose dimension is…

Numerical Analysis · Computer Science 2014-11-04 Mostafa Rahmani , George Atia

A new class of disturbance covariance matrix estimators for radar signal processing applications is introduced following a geometric paradigm. Each estimator is associated with a given unitary invariant norm and performs the sample…

Applications · Statistics 2018-02-14 Augusto Aubry , Antonio De Maio , Luca Pallotta

This chapter reviews methods for linear shrinkage of the sample covariance matrix (SCM) and matrices (SCM-s) under elliptical distributions in single and multiple populations settings, respectively. In the single sample setting a popular…

Methodology · Statistics 2023-08-10 Esa Ollila

Relying on recent advances in statistical estimation of covariance distances based on random matrix theory, this article proposes an improved covariance and precision matrix estimation for a wide family of metrics. The method is shown to…

Machine Learning · Statistics 2021-02-03 Malik Tiomoko , Florent Bouchard , Guillaume Ginholac , Romain Couillet

Robustness to outliers is often a desirable property of statistical estimators. Indeed many well known estimators offer very good optimal performance in theory but are unusable in applied contexts because of their sensitivity to outliers.…

Statistics Theory · Mathematics 2016-12-01 Christophe Culan , Claude Adnet

Consider estimating the n by p matrix of means of an n by p matrix of independent normally distributed observations with constant variance, where the performance of an estimator is judged using a p by p matrix quadratic error loss function.…

Statistics Theory · Mathematics 2011-01-19 Reman Abu-Shanab , John T. Kent , William E. Strawderman

This paper focuses on Bayesian shrinkage for covariance matrix estimation. We examine posterior properties and frequentist risks of Bayesian estimators based on new hierarchical inverse-Wishart priors. More precisely, we give the existence…

Methodology · Statistics 2011-06-17 Mathilde Bouriga , Olivier Féron

We describe an efficient algorithm for calculating the statistics of weak lensing by large-scale structure based on a tiled set of independent particle-mesh N-body simulations which telescope in resolution along the line of sight. This…

Astrophysics · Physics 2008-11-26 Martin White , Wayne Hu

In large-scale, data-driven applications, parameters are often only known approximately due to noise and limited data samples. In this paper, we focus on high-dimensional optimization problems with linear constraints under uncertain…

Optimization and Control · Mathematics 2024-03-01 Naqi Huang , Nestor Parolya , Theresia van Essen

Standard noise radars, as well as noise-type radars such as quantum two-mode squeezing radar, are characterized by a covariance matrix with a very specific structure. This matrix has four independent parameters: the amplitude of the…

Signal Processing · Electrical Eng. & Systems 2022-08-10 David Luong , Bhashyam Balaji , Sreeraman Rajan

In this paper we investigate the performance of periodogram based estimators of the spectral density matrix of possibly high-dimensional time series. We suggest and study shrinkage as a remedy against numerical instabilities due to…

Statistics Theory · Mathematics 2008-08-13 Hilmar Böhm , Rainer von Sachs

Large datasets are often affected by cell-wise outliers in the form of missing or erroneous data. However, discarding any samples containing outliers may result in a dataset that is too small to accurately estimate the covariance matrix.…

Statistics Theory · Mathematics 2023-11-13 Karim Lounici , Grégoire Pacreau

This manuscript presents an approach to perform generalized linear regression with multiple high dimensional covariance matrices as the outcome. Model parameters are proposed to be estimated by maximizing a pseudo-likelihood. When the data…

Methodology · Statistics 2020-07-28 Yi Zhao , Brian S. Caffo , Xi Luo