Related papers: Cleaning large correlation matrices: tools from ra…
Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be…
We present a comparison between various algorithms of inference of covariance and precision matrices in small datasets of real vectors, of the typical length and dimension of human brain activity time series retrieved by functional Magnetic…
Cosmological parameter inference from galaxy clustering relies critically on accurate estimates of the covariance and precision matrices. These are often obtained from a limited number of mock catalogs, introducing noise and bias in the…
Covariance matrices estimated from short, noisy, and non-Gaussian financial time series are notoriously unstable. Empirical evidence suggests that such covariance structures often exhibit power-law scaling, reflecting complex, hierarchical…
Scattering of electromagnetic waves in billiard-like systems has become a standard experimental tool of studying properties associated with Quantum Chaos. Random Matrix Theory (RMT) describing statistics of eigenfrequencies and associated…
Portfolio theory is a very powerful tool in the modern investment theory. It is helpful in estimating risk of an investor's portfolio, which arises from our lack of information, uncertainty and incomplete knowledge of reality, which forbids…
Recently, Chernozhukov, Chetverikov, and Kato [Ann. Statist. 42 (2014) 1564--1597] developed a new Gaussian comparison inequality for approximating the suprema of empirical processes. This paper exploits this technique to devise sharp…
Simulating sample correlation matrices is important in many areas of statistics. Approaches such as generating Gaussian data and finding their sample correlation matrix or generating random uniform $[-1,1]$ deviates as pairwise correlations…
We show that results from the theory of random matrices are potentially of great interest to understand the statistical structure of the empirical correlation matrices appearing in the study of price fluctuations. The central result of the…
We consider estimating a matrix from noisy observations coming from an arbitrary additive bi-rotational invariant perturbation. We propose an estimator which is optimal among the class of rectangular rotational invariant estimators and can…
Modern technologies are producing datasets with complex intrinsic structures, and they can be naturally represented as matrices instead of vectors. To preserve the latent data structures during processing, modern regression approaches…
In dealing with high-dimensional data sets, factor models are often useful for dimension reduction. The estimation of factor models has been actively studied in various fields. In the first part of this paper, we present a new approach to…
Through simple analytical calculations and numerical simulations, we demonstrate the generic existence of a self-organized macroscopic state in any large multivariate system possessing non-vanishing average correlations between a finite…
The use of sparse precision (inverse covariance) matrices has become popular because they allow for efficient algorithms for joint inference in high-dimensional models. Many applications require the computation of certain elements of the…
Random matrix theory allows one to deduce the eigenvalue spectrum of a large matrix given only statistical information about its elements. Such results provide insight into what factors contribute to the stability of complex dynamical…
Often in applications ranging from medical imaging and sensor networks to error correction and data science (and beyond), one needs to solve large-scale linear systems in which a fraction of the measurements have been corrupted. We consider…
Matching is one of the simplest approaches for estimating causal effects from observational data. Matching techniques compare the observed outcomes across pairs of individuals with similar covariate values but different treatment statuses…
Covariance matrix estimation is a persistent challenge for cosmology. We focus on a class of model covariance matrices that can be generated with high accuracy and precision, using a tiny fraction of the computational resources that would…
Financial stock returns correlations have been studied in the prism of random matrix theory, to distinguish the signal from the "noise". Eigenvalues of the matrix that are above the rescaled Marchenko Pastur distribution can be interpreted…
Confounding is a significant obstacle to unbiased estimation of causal effects from observational data. For settings with high-dimensional covariates -- such as text data, genomics, or the behavioral social sciences -- researchers have…