English
Related papers

Related papers: New Methods for Handling Singular Sample Covarianc…

200 papers

We consider the problem of learning error covariance matrices for robotic state estimation. The convergence of a state estimator to the correct belief over the robot state is dependent on the proper tuning of noise models. During inference,…

Robotics · Computer Science 2023-09-19 Mohamad Qadri , Zachary Manchester , Michael Kaess

This paper presents how the most recent improvements made on covariance matrix estimation and model order selection can be applied to the portfolio optimisation problem. The particular case of the Maximum Variety Portfolio is treated but…

Applications · Statistics 2018-04-03 Emmanuelle Jay , Eugénie Terreaux , Jean-Philippe Ovarlez , Frédéric Pascal

Low-rank matrix approximations are often used to help scale standard machine learning algorithms to large-scale problems. Recently, matrix coherence has been used to characterize the ability to extract global information from a subset of…

Machine Learning · Statistics 2010-09-07 Mehryar Mohri , Ameet Talwalkar

The Ewens sampling formula is a distribution related to the random partition of a positive integer. In this study, we investigate the issue of non-existence solutions in parameter estimation under the distribution. As a result, the first…

Statistics Theory · Mathematics 2021-05-25 Masayo Y. Hirose , Shuhei Mano

The survey experiment is widely used in economics and social sciences to evaluate the effects of treatments or programs. In a standard population-based survey experiment, the experimenter randomly draws experimental units from a target…

Methodology · Statistics 2026-05-11 Pengfei Tian , Jiyang Ren , Yingying Ma

The present work provides an original framework for random matrix analysis based on revisiting the concentration of measure theory from a probabilistic point of view. By providing various notions of vector concentration ($q$-exponential,…

Probability · Mathematics 2021-01-19 Cosme Louart , Romain Couillet

Covariate-adaptive randomization is widely used in clinical trials to balance prognostic factors, and regression adjustments are often adopted to further enhance the estimation and inference efficiency. In practice, the covariates may…

Methodology · Statistics 2025-08-15 Wanjia Fu , Yingying Ma , Hanzhong Liu

This paper tackles the problem of robust covariance matrix estimation when the data is incomplete. Classical statistical estimation methodologies are usually built upon the Gaussian assumption, whereas existing robust estimation ones assume…

A major problem in numerical weather prediction (NWP) is the estimation of high-dimensional covariance matrices from a small number of samples. Maximum likelihood estimators cannot provide reliable estimates when the overall dimension is…

Methodology · Statistics 2023-01-13 Robert J. Webber , Matthias Morzfeld

This paper focuses on the estimation of the sample covariance matrix from low-dimensional random projections of data known as compressive measurements. In particular, we present an unbiased estimator to extract the covariance structure from…

Machine Learning · Statistics 2017-05-01 Farhad Pourkamali-Anaraki

The density function of the limiting spectral distribution of general sample covariance matrices is usually unknown. We propose to use kernel estimators which are proved to be consistent. A simulation study is also conducted to show the…

Statistics Theory · Mathematics 2012-11-15 Bing-Yi Jing , Guangming Pan , Qi-Man Shao , Wang Zhou

We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a $0-1$…

Machine Learning · Statistics 2018-04-06 Eduardo Pavez , Antonio Ortega

Covariance matrix estimation arises in multivariate problems including multivariate normal sampling models and regression models where random effects are jointly modeled, e.g. random-intercept, random-slope models. A Bayesian analysis of…

Methodology · Statistics 2016-07-14 Ignacio Alvarez , Jarad Niemi , Matt Simpson

We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a…

Methodology · Statistics 2019-10-03 Aviv Navon , Saharon Rosset

We seek to improve estimates of the power spectrum covariance matrix from a limited number of simulations by employing a novel statistical technique known as shrinkage estimation. The shrinkage technique optimally combines an empirical…

Astrophysics · Physics 2009-11-13 Adrian C. Pope , István Szapudi

Estimating the diagonal entries of a matrix, that is not directly accessible but only available as a linear operator in the form of a computer routine, is a common necessity in many computational applications, especially in image…

Instrumentation and Methods for Astrophysics · Physics 2015-03-19 Marco Selig , Niels Oppermann , Torsten A. Enßlin

The Welch-Satterthwaite t-test is one of the most prominent and often used statistical inference method in applications. The method is, however, not flexible with respect to adjustments for baseline values or other covariates, which may…

Methodology · Statistics 2018-08-29 Cong Cao , Markus Pauly , Frank Konietschke

The crossed random effects model is widely used, finding applications in various fields such as longitudinal studies, e-commerce, and recommender systems, among others. However, these models encounter scalability challenges, as the…

Methodology · Statistics 2025-10-21 Disha Ghandwani , Swarnadip Ghosh , Trevor Hastie , Art B. Owen

We consider estimating the population covariance matrix when the number of available samples is less than the size of the observations. The sample covariance matrix (SCM) being singular, regularization is mandatory in this case. For this…

Statistics Theory · Mathematics 2025-06-16 Olivier Besson

In this article we study in detail a family of random matrix ensembles which are obtained from random permutations matrices (chosen at random according to the Ewens measure of parameter $\theta>0$) by replacing the entries equal to one by…

Probability · Mathematics 2010-05-05 Joseph Najnudel , Ashkan Nikeghbali