English
Related papers

Related papers: Some Statistical Problems with High Dimensional Fi…

200 papers

High dimensional statistical problems arise from diverse fields of scientific research and technological development. Variable selection plays a pivotal role in contemporary statistical learning and scientific discoveries. The traditional…

Statistics Theory · Mathematics 2009-10-08 Jianqing Fan , Jinchi Lv

The sample covariance matrix becomes non-invertible in high-dimensional settings, making classical multivariate statistical methods inapplicable. Various regularization techniques address this issue by imposing a structured target matrix to…

Methodology · Statistics 2025-03-13 Atiq Ur Rehman , Muhammad Farooq

Learning in the presence of outliers is a fundamental problem in statistics. Until recently, all known efficient unsupervised learning algorithms were very sensitive to outliers in high dimensions. In particular, even for the task of robust…

Data Structures and Algorithms · Computer Science 2019-11-15 Ilias Diakonikolas , Daniel M. Kane

High-dimensional time series datasets are becoming increasingly common in many areas of biological and social sciences. Some important applications include gene regulatory network reconstruction using time course gene expression data, brain…

Methodology · Statistics 2021-08-02 Sumanta Basu , David S. Matteson

Robust statistics aims to compute quantities to represent data where a fraction of it may be arbitrarily corrupted. The most essential statistic is the mean, and in recent years, there has been a flurry of theoretical advancement for…

Machine Learning · Statistics 2025-02-18 Cullen Anderson , Jeff M. Phillips

This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high-dimensional models? In particular, we look at the error rates and power of some multi-stage regression…

Statistics Theory · Mathematics 2009-08-20 Larry Wasserman , Kathryn Roeder

Robust estimation is much more challenging in high dimensions than it is in one dimension: Most techniques either lead to intractable optimization problems or estimators that can tolerate only a tiny fraction of errors. Recent work in…

Machine Learning · Computer Science 2018-03-14 Ilias Diakonikolas , Gautam Kamath , Daniel M. Kane , Jerry Li , Ankur Moitra , Alistair Stewart

Estimation of high dimensional covariance matrices is an interesting and important research topic. In this paper, we propose a dynamic structure and develop an estimation procedure for high dimensional covariance matrices. Asymptotic…

Methodology · Statistics 2015-06-05 Shaojun Guo , John Box , Wenyang Zhang

These lecture notes provide an overview of existing methodologies and recent developments for estimation and inference with high dimensional time series regression models. First, we present main limit theory results for high dimensional…

Econometrics · Economics 2023-09-01 Christis Katsouris

It is clear that conventional statistical inference protocols need to be revised to deal correctly with the high-dimensional data that are now common. Most recent studies aimed at achieving this revision rely on powerful approximation…

Statistics Theory · Mathematics 2021-05-05 Alexander Mozeika , Mansoor Sheikh , Fabian Aguirre-Lopez , Fabrizio Antenucci , Anthony CC Coolen

Simulated high-dimensional data is useful for testing, validating, and improving algorithms used in dimension reduction, supervised and unsupervised learning. High-dimensional data is characterized by multiple variables that are dependent…

This paper considers the problem of estimating a change point in the covariance matrix in a sequence of high-dimensional vectors, where the dimension is substantially larger than the sample size. A two-stage approach is proposed to…

Methodology · Statistics 2018-07-31 H. Dette , G. M. Pan , Q. Yang

Regression analysis is a key area of interest in the field of data analysis and machine learning which is devoted to exploring the dependencies between variables, often using vectors. The emergence of high dimensional data in technologies…

Machine Learning · Statistics 2023-08-23 Jiani Liu , Ce Zhu , Zhen Long , Yipeng Liu

High dimensionality comparable to sample size is common in many statistical problems. We examine covariance matrix estimation in the asymptotic framework that the dimensionality $p$ tends to $\infty$ as the sample size $n$ increases.…

Statistics Theory · Mathematics 2007-06-13 Jianqing Fan , Yingying Fan , Jinchi Lv

The utilization of statistical methods an their applications within the new field of study known as Topological Data Analysis has has tremendous potential for broadening our exploration and understanding of complex, high-dimensional data…

Applications · Statistics 2016-07-19 Patrick S. Medina , R. W. Doerge

Regression is one of the most fundamental statistical inference problems. A broad definition of regression problems is as estimation of the distribution of an outcome using a family of probability models indexed by covariates. Despite the…

Statistics Theory · Mathematics 2023-09-26 Peter Mueller , Fernando Andrés Quintana , Garritt L. Page

This paper studies the case of possibly high-dimensional covariates in the regression discontinuity design (RDD) analysis. In particular, we propose estimation and inference methods for the RDD models with covariate selection which perform…

Econometrics · Economics 2026-01-21 Yoichi Arai , Taisuke Otsu , Myung Hwan Seo

The present paper provides a study of high-dimensional statistical arbitrage that combines factor models with the tools from stochastic control, obtaining closed-form optimal strategies which are both interpretable and computationally…

Mathematical Finance · Quantitative Finance 2021-06-25 Jorge Guijarro-Ordonez

The problem of detecting changes in covariance for a single pair of features has been studied in some detail, but may be limited in importance or general applicability. In contrast, testing equality of covariance matrices of a {\it set} of…

Methodology · Statistics 2017-12-12 Yi-Hui Zhou

Current statistical inference problems in areas like astronomy, genomics, and marketing routinely involve the simultaneous testing of thousands -- even millions -- of null hypotheses. For high-dimensional multivariate distributions, these…

Methodology · Statistics 2017-04-25 Weixin Cai , Nima S. Hejazi , Alan E. Hubbard