English
Related papers

Related papers: Optimal High-Dimensional Shrinkage Covariance Esti…

200 papers

Despite the fast advances in high-sigma yield analysis with the help of machine learning techniques in the past decade, one of the main challenges, the curse of dimensionality, which is inevitable when dealing with modern large-scale…

Computational Engineering, Finance, and Science · Computer Science 2022-12-06 Shuo Yin , Guohao Dai , Wei W. Xing

Covariance matrix estimation is an important problem in multivariate data analysis, both from theoretical as well as applied points of view. Many simple and popular covariance matrix estimators are known to be severely affected by model…

Methodology · Statistics 2025-11-21 Soumya Chakraborty , Ayanendranath Basu , Abhik Ghosh

Support vector machine (SVM) has been one of the most popular learning algorithms, with the central idea of maximizing the minimum margin, i.e., the smallest distance from the instances to the classification boundary. Recent theoretical…

Machine Learning · Computer Science 2020-07-07 Teng Zhang , Zhi-Hua Zhou

Stochastic PDE eigenvalue problems often arise in the field of uncertainty quantification, whereby one seeks to quantify the uncertainty in an eigenvalue, or its eigenfunction. In this paper we present an efficient multilevel quasi-Monte…

Numerical Analysis · Mathematics 2022-10-07 Alexander D. Gilbert , Robert Scheichl

This paper considers the problem of robustly estimating a structured covariance matrix with an elliptical underlying distribution with known mean. In applications where the covariance matrix naturally possesses a certain structure, taking…

Applications · Statistics 2016-06-29 Ying Sun , Prabhu Babu , Daniel P. Palomar

We consider distributed estimation of the inverse covariance matrix, also called the concentration or precision matrix, in Gaussian graphical models. Traditional centralized estimation often requires global inference of the covariance…

Machine Learning · Statistics 2015-06-15 Zhaoshi Meng , Dennis Wei , Ami Wiesel , Alfred O. Hero

In this paper, we consider the problem of determining the presence of a given signal in a high-dimensional observation with unknown covariance matrix by using an adaptive matched filter. Traditionally such filters are formed from the sample…

Statistics Theory · Mathematics 2021-12-06 Benjamin D. Robinson , Robert Malinas , Alfred O. Hero

Machine learning algorithms in high-dimensional settings are highly susceptible to the influence of even a small fraction of structured outliers, making robust optimization techniques essential. In particular, within the…

Machine Learning · Computer Science 2025-04-25 Changyu Gao , Andrew Lowy , Xingyu Zhou , Stephen J. Wright

This paper proposes a distributed alternating mixed discrete-continuous (DAMDC) algorithm to approach the oracle algorithm based on the diffusion strategy for parameter and spectrum estimation over sensor networks. A least mean squares…

Distributed, Parallel, and Cluster Computing · Computer Science 2016-04-12 R. C. de Lamare

A new robust correlation estimator based on the spatial sign covariance matrix (SSCM) is proposed. We derive its asymptotic distribution and influence function at elliptical distributions. Finite sample and robustness properties are studied…

Methodology · Statistics 2022-04-12 Alexander Dürre , Daniel Vogel , Roland Fried

Markov Chain Monte Carlo (MCMC) sampler is widely used for cosmological parameter estimation from CMB and other data. However, due to the intrinsic serial nature of the MCMC sampler, convergence is often very slow. Here we present a fast…

Cosmology and Nongalactic Astrophysics · Physics 2014-08-12 Santanu Das , Tarun Souradeep

We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a $p$-dimensional Gaussian random vector from $n$ independent samples. The proposed model…

Optimization and Control · Mathematics 2018-05-21 Viet Anh Nguyen , Daniel Kuhn , Peyman Mohajerin Esfahani

In a linear regression model with random design, we consider a family of candidate models from which we want to select a `good' model for prediction out-of-sample. We fit the models using block shrinkage estimators, and we focus on the…

Statistics Theory · Mathematics 2018-09-13 Hannes Leeb , Nina Senitschnig

Unlike the ordinary least-squares (OLS) estimator for the linear model, a ridge regression linear model provides coefficient estimates via shrinkage, usually with improved mean-square and prediction error. This is true especially when the…

Methodology · Statistics 2015-06-25 George Karabatsos

Multi-fidelity methods that use an ensemble of models to compute a Monte Carlo estimator of the expectation of a high-fidelity model can significantly reduce computational costs compared to single-model approaches. These methods use oracle…

Computation · Statistics 2026-03-12 Thomas Dixon , Alex Gorodetsky , John Jakeman , Akil Narayan , Yiming Xu

In this article we provide some nonnegative and positive estimators of the mean squared errors(MSEs) for shrinkage estimators of multivariate normal means. Proposed estimators are shown to improve on the uniformly minimum variance unbiased…

Statistics Theory · Mathematics 2007-10-08 Hisayuki Hara

High-dimensional covariance estimation is notoriously sensitive to outliers. While statistically optimal estimators exist for general heavy-tailed distributions, they often rely on computationally expensive techniques like semidefinite…

Machine Learning · Statistics 2026-01-06 Even He

We study shrinkage estimation of the mean parameters of a class of multivariate distributions for which the diagonal entries of the corresponding covariance matrix are certain quadratic functions of the mean parameter. This class of…

Statistics Theory · Mathematics 2022-07-04 Nikolas Siapoutis , Donald Richards , Bharath K. Sriperumbudur

This paper considers the problem of optimizing the average tracking error for an elliptic partial differential equation with an uncertain lognormal diffusion coefficient. In particular, the application of the multilevel quasi-Monte Carlo…

Numerical Analysis · Mathematics 2021-09-30 Philipp A. Guth , Andreas Van Barel

The application of standard sufficient dimension reduction methods for reducing the dimension space of predictors without losing regression information requires inverting the covariance matrix of the predictors. This has posed a number of…

Methodology · Statistics 2019-10-01 Kabir Opeyemi Olorede , Waheed Babatunde Yahya