English
Related papers

Related papers: Testing Endogeneity with High Dimensional Covariat…

200 papers

In this study, we prove that an intrinsic low dimensionality of covariates is the main factor that determines the performance of deep neural networks (DNNs). DNNs generally provide outstanding empirical performance. Hence, numerous studies…

Machine Learning · Statistics 2020-09-18 Ryumei Nakada , Masaaki Imaizumi

Traditional instrumental variable (IV) estimators face a fundamental constraint: they can only accommodate as many endogenous treatment variables as available instruments. This limitation becomes particularly challenging in settings where…

Machine Learning · Computer Science 2025-06-25 Shiangyi Lin , Hui Lan , Vasilis Syrgkanis

In this article, we propose some two-sample tests based on ball divergence and investigate their high dimensional behavior. First, we study their behavior for High Dimension, Low Sample Size (HDLSS) data, and under appropriate regularity…

Statistics Theory · Mathematics 2024-10-08 Bilol Banerjee , Anil K. Ghosh

In this paper, we study the problem of testing the mean vectors of high dimensional data in both one-sample and two-sample cases. The proposed testing procedures employ maximum-type statistics and the parametric bootstrap techniques to…

Statistics Theory · Mathematics 2018-01-23 Jinyuan Chang , Chao Zheng , Wen-Xin Zhou , Wen Zhou

When applying multivariate extreme value statistics to analyze tail risk in compound events defined by a multivariate random vector, one often assumes that all dimensions share the same extreme value index. While such an assumption can be…

Methodology · Statistics 2026-02-16 Liujun Chen , Chen Zhou

Many statistical methodologies for high-dimensional data assume the population is normal. Although a few multivariate normality tests have been proposed, to the best of our knowledge, none of them can properly control the type I error when…

Methodology · Statistics 2021-05-04 Hao Chen , Yin Xia

Background: High-dimensional genomic data exhibit strong group correlation structures that challenge conventional feature selection methods, which often assume feature independence or rely on pre-defined pathways and are sensitive to…

Machine Learning · Computer Science 2026-02-10 Wanru Guo , Juan Xie , Binbin Wang , Weicong Chen , Xiaoyi Lu , Vipin Chaudhary , Curtis Tatsuoka

We propose a two-sample test for covariance matrices in the high-dimensional regime, where the dimension diverges proportionally to the sample size. Our hybrid test combines a Frobenius-norm-based statistic as considered in Li and Chen…

Statistics Theory · Mathematics 2025-06-10 Thomas Lam , Nina Dörnemann , Holger Dette

Recently, applied sciences, including longitudinal and clustered studies in biomedicine require the analysis of ultra-high dimensional linear mixed effects models where we need to select important fixed effect variables from a vast pool of…

Methodology · Statistics 2020-05-01 Abhik Ghosh , Magne Thoresen

We propose a new testing procedure of heteroskedasticity in high-dimensional linear regression, where the number of covariates can be larger than the sample size. Our testing procedure is based on residuals of the Lasso. We demonstrate that…

Statistics Theory · Mathematics 2022-11-01 Akira Shinkyu

Most papers on high-dimensional statistics are based on the assumption that none of the regressors are correlated with the regression error, namely, they are exogenous. Yet, endogeneity can arise incidentally from a large pool of regressors…

Statistics Theory · Mathematics 2014-05-28 Jianqing Fan , Yuan Liao

In this paper, we investigate score function-based tests to check the significance of an ultrahigh-dimensional sub-vector of the model coefficients when the nuisance parameter vector is also ultrahigh-dimensional in linear models. We first…

Methodology · Statistics 2024-11-12 Weichao Yang , Xu Guo , Lixing Zhu

We propose a new model specification test for multiple-equation systems with cross-equation error and dynamic regressor--error dependences. Conventional tests often rely on exogeneity conditions strong enough to ensure consistency of the…

Econometrics · Economics 2026-04-22 Koichiro Moriya , Akihiko Noda

The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models, are among the most challenging problems in statistical practice. These challenges can be tackled using a…

Methodology · Statistics 2022-04-06 Sara Algeri , David A. van Dyk

High dimensional hypothesis test deals with models in which the number of parameters is significantly larger than the sample size. Existing literature develops a variety of individual tests. Some of them are sensitive to the dense and small…

Statistics Theory · Mathematics 2018-08-09 Cheng Zhou , Xinsheng Zhang , Wenxin Zhou , Han Liu

Testing for the equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been…

Methodology · Statistics 2019-11-22 Soham Sarkar , Rahul Biswas , Anil K. Ghosh

This paper presents a selective survey of recent developments in statistical inference and multiple testing for high-dimensional regression models, including linear and logistic regression. We examine the construction of confidence…

Methodology · Statistics 2023-01-26 T. Tony Cai , Zijian Guo , Yin Xia

High dimension low sample size statistical analysis is important in a wide range of applications. In such situations, the highly appealing discrimination method, support vector machine, can be improved to alleviate data piling at the…

Optimization and Control · Mathematics 2017-08-18 Xin Yee Lam , J. S. Marron , Defeng Sun , Kim-Chuan Toh

This article deals with the problem of testing conditional independence between two random vectors ${\bf X}$ and ${\bf Y}$ given a confounding random vector ${\bf Z}$. Several authors have considered this problem for multivariate data.…

Statistics Theory · Mathematics 2025-09-16 Bilol Banerjee

The Wilcoxon-Mann-Whitney test is a robust competitor of the t-test in the univariate setting. For finite dimensional multivariate data, several extensions of the Wilcoxon-Mann-Whitney test have been shown to have better performance than…

Methodology · Statistics 2014-03-04 Anirvan Chakraborty , Probal Chaudhuri