Related papers: A fast and reliable method for the comparison of c…
When inferring parameters from a Gaussian-distributed data set by computing a likelihood, a covariance matrix is needed that describes the data errors and their correlations. If the covariance matrix is not known a priori, it may be…
Statistical inference and information processing of high-dimensional data often require efficient and accurate estimation of their second-order statistics. With rapidly changing data, limited processing power and storage at the acquisition…
A separable covariance model for a random matrix provides a parsimonious description of the covariances among the rows and among the columns of the matrix, and permits likelihood-based inference with a very small sample size. However, in…
We investigate, in dark matter and galaxy mocks, the effects of approximating the galaxy power spectrum-bispectrum estimated covariance as a diagonal matrix, for an analysis that aligns with the specifications of recent and upcoming galaxy…
The nonlinear mass is a characteristic scale in halo formation that has wide-ranging applications across cosmology. Naively, computing it requires repeated numerical integration to calculate the variance of the power spectrum on different…
Making cosmological inferences from the observed galaxy clustering requires accurate predictions for the mean clustering statistics and their covariances. Those are affected by cosmic variance -- the statistical noise due to the finite…
Covariance matrix estimation and principal component analysis (PCA) are two cornerstones of multivariate analysis. Classic textbook solutions perform poorly when the dimension of the data is of a magnitude similar to the sample size, or…
By introducing Crossing functions and hyper-parameters I show that the Bayesian interpretation of the Crossing Statistics [1] can be used trivially for the purpose of model selection among cosmological models. In this approach to falsify a…
We present a method for estimating sparse high-dimensional inverse covariance and partial correlation matrices, which exploits the connection between the inverse covariance matrix and linear regression. The method is a two-stage estimation…
This paper deals with the problem of estimating the covariance matrix of a series of independent multivariate observations, in the case where the dimension of each observation is of the same order as the number of observations. Although…
One of the goals in scaling sequential machine learning methods pertains to dealing with high-dimensional data spaces. A key related challenge is that many methods heavily depend on obtaining the inverse covariance matrix of the data. It is…
Weak gravitational lensing is one of the few direct methods to map the dark-matter distribution on large scales in the Universe, and to estimate cosmological parameters. We study a Bayesian inference problem where the data covariance…
One of the primary sources of uncertainties in modeling the cosmic-shear power spectrum on small scales is the effect of baryonic physics. Accurate cosmology for Stage-IV surveys requires knowledge of the matter power spectrum deep in the…
This paper tackles the problem of robust covariance matrix estimation when the data is incomplete. Classical statistical estimation methodologies are usually built upon the Gaussian assumption, whereas existing robust estimation ones assume…
Linear regression models depend directly on the design matrix and its properties. Techniques that efficiently estimate model coefficients by partitioning rows of the design matrix are increasingly popular for large-scale problems because…
We introduce a new conservative test for quantifying the consistency of two or more datasets. The test is based on the Bayesian answer to the question, ``How much more probable is it that all my data were generated from the same model…
We describe an approximate statistical model for the sample variance distribution of the non-linear matter power spectrum that can be calibrated from limited numbers of simulations. Our model retains the common assumption of a multivariate…
We apply two Bayesian hierarchical inference schemes to infer shear power spectra, shear maps and cosmological parameters from the CFHTLenS weak lensing survey - the first application of this method to data. In the first approach, we sample…
Statistical analysis of massive datasets very often implies expensive linear algebra operations with large dense matrices. Typical tasks are an estimation of unknown parameters of the underlying statistical model and prediction of missing…
This article presents a new method to compute matrices from numerical simulations based on the ideas of sparse sampling and compressed sensing. The method is useful for problems where the determination of the entries of a matrix constitutes…