English
Related papers

Related papers: Cross-validated covariance estimators for high-dim…

200 papers

K-fold cross validation (CV) is a popular method for estimating the true performance of machine learning models, allowing model selection and parameter tuning. However, the very process of CV requires random partitioning of the data and so…

Computation and Language · Computer Science 2018-06-20 Henry B. Moss , David S. Leslie , Paul Rayson

The majority of standard approaches to financial portfolio optimization (PO) are based on the mean-variance (MV) framework. Given a risk aversion coefficient, the MV procedure yields a single portfolio that represents the optimal trade-off…

Portfolio Management · Quantitative Finance 2024-02-27 Bruno Gašperov , Marko Đurasević , Domagoj Jakobovic

In this paper we consider estimation of sparse covariance matrices and propose a thresholding procedure which is adaptive to the variability of individual entries. The estimators are fully data driven and enjoy excellent performance both…

Methodology · Statistics 2011-02-14 Tony Cai , Weidong Liu

This study explores the use of Transformer-based models to predict both covariance and semi-covariance matrices for ETF portfolio optimization. Traditional portfolio optimization techniques often rely on static covariance estimates or…

Portfolio Management · Quantitative Finance 2024-12-02 Jiahao Zhu , Hengzhi Wu

Cross-validation (CV) is widely used for tuning a model with respect to user-selected parameters and for selecting a "best" model. For example, the method of $k$-nearest neighbors requires the user to choose $k$, the number of neighbors,…

Applications · Statistics 2012-03-01 Hui Shen , William J. Welch , Jacqueline M. Hughes-Oliver

Portfolio optimization aims at constructing a realistic portfolio with significant out-of-sample performance, which is typically measured by the out-of-sample Sharpe ratio. However, due to in-sample optimism, it is inappropriate to use the…

Statistics Theory · Mathematics 2025-07-11 Xuran Meng , Yuan Cao , Weichen Wang

Covariance matrix estimation is a fundamental statistical task in many applications, but the sample covariance matrix is sub-optimal when the sample size is comparable to or less than the number of features. Such high-dimensional settings…

Methodology · Statistics 2022-06-06 Huiqin Xin , Sihai Dave Zhao

This paper is concerned with portfolio optimization models for creating high-quality lists of recommended items to balance the accuracy and diversity of recommendations. However, the statistics (i.e., expectation and covariance of ratings)…

Information Retrieval · Computer Science 2024-10-01 Tomoya Yanagi , Shunnosuke Ikeda , Yuichi Takano

We discuss a weighted estimation of correlation and covariance matrices from historical financial data. To this end, we introduce a weighting scheme that accounts for similarity of previous market conditions to the present one. The…

Statistical Finance · Quantitative Finance 2010-07-01 Michael C. Münnix , Rudi Schäfer , Oliver Grothe

We consider the problem of estimating the parameters of the covariance function of a Gaussian process by cross-validation. We suggest using new cross-validation criteria derived from the literature of scoring rules. We also provide an…

Computation · Statistics 2020-08-07 Sébastien Petit , Julien Bect , Sébastien da Veiga , Paul Feliot , Emmanuel Vazquez

Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on…

Machine Learning · Statistics 2016-11-04 Daniel Bartz

Recent studies stressed the fact that covariance matrices computed from empirical financial time series appear to contain a high amount of noise. This makes the classical Markowitz Mean-Variance Optimization model unable to correctly…

Optimization and Control · Mathematics 2021-03-03 Justo Puerto , Federica Ricca , Moisés Rodríguez-Madrena , Andrea Scozzari

Cross-validation is frequently used for model selection in a variety of applications. However, it is difficult to apply cross-validation to mixed effects models (including nonlinear mixed effects models or NLME models) due to the fact that…

Methodology · Statistics 2013-05-24 Emily Colby , Eric Bair

This paper tackles the problem of robust covariance matrix estimation when the data is incomplete. Classical statistical estimation methodologies are usually built upon the Gaussian assumption, whereas existing robust estimation ones assume…

Cross-validation plays a fundamental role in Machine Learning, enabling robust evaluation of model performance and preventing overestimation on training and validation data. However, one of its drawbacks is the potential to create data…

Machine Learning · Computer Science 2025-08-28 Afonso Martini Spezia , Thomas Fontanari , Mariana Recamonde-Mendoza

High-dimensional covariance estimation is notoriously sensitive to outliers. While statistically optimal estimators exist for general heavy-tailed distributions, they often rely on computationally expensive techniques like semidefinite…

Machine Learning · Statistics 2026-01-06 Even He

Modeling and managing portfolio risk is perhaps the most important step to achieve growing and preserving investment performance. Within the modern portfolio construction framework that built on Markowitz's theory, the covariance matrix of…

Risk Management · Quantitative Finance 2021-10-28 Hengxu Lin , Dong Zhou , Weiqing Liu , Jiang Bian

Cross-validation is a standard tool for obtaining a honest assessment of the performance of a prediction model. The commonly used version repeatedly splits data, trains the prediction model on the training set, evaluates the model…

Machine Learning · Statistics 2025-10-10 Tianyu Pan , Vincent Z. Yu , Viswanath Devanarayan , Lu Tian

We consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero. We first present an algorithm, which we call Iterative Conditional Fitting, for computing the maximum…

Statistics Theory · Mathematics 2010-03-04 Sanjay Chaudhuri , Mathias Drton , Thomas S. Richardson

In the realm of high-dimensional data analysis, the estimation of covariance matrices is a fundamental task, and this holds true for interval-valued data as well. However, there is no unified definition for the covariance matrix of…

Methodology · Statistics 2026-04-02 Wan Tian , Wenhao Cui , Rui Zhang , Bingyi Jing , Yang Liu , Yijie Peng