English
Related papers

Related papers: Regularization after retention in ultrahigh dimens…

200 papers

In data sets with many more features than observations, independent screening based on all univariate regression models leads to a computationally convenient variable selection method. Recent efforts have shown that in the case of…

Machine Learning · Statistics 2011-08-12 Anders Gorst-Rasmussen , Thomas H. Scheike

Variable screening has been a useful research area that deals with ultrahigh-dimensional data. When there exist both marginally and jointly dependent predictors to the response, existing methods such as conditional screening or iterative…

Methodology · Statistics 2023-07-10 Lei Fang , Qingcong Yuan , Xiangrong Yin , Chenglong Ye

We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the…

Statistics Theory · Mathematics 2014-11-19 Min Xu , Minhua Chen , John Lafferty

We consider the problem of variable screening in ultra-high dimensional generalized linear models (GLMs) of non-polynomial orders. Since the popular SIS approach is extremely unstable in the presence of contamination and noise, we discuss a…

Statistics Theory · Mathematics 2022-11-15 Abhik Ghosh , Erica Ponzi , Torkjel Sandanger , Magne Thoresen

We introduce a quantile-adaptive framework for nonlinear variable screening with high-dimensional heterogeneous data. This framework has two distinctive features: (1) it allows the set of active variables to vary across quantiles, thus…

Statistics Theory · Mathematics 2013-12-12 Xuming He , Lan Wang , Hyokyoung Grace Hong

In this paper we propose a linear variable screening method for computer experiments when the number of input variables is larger than the number of runs. This method uses a linear model to model the nonlinear data, and screens the…

Methodology · Statistics 2020-06-16 Chunya Li , Daijun Chen , Shifeng Xiong

Independence screening is a powerful method for variable selection for `Big Data' when the number of variables is massive. Commonly used independence screening methods are based on marginal correlations or variations of it. In many…

Statistics Theory · Mathematics 2012-11-02 Emre Barut , Jianqing Fan , Anneleen Verhasselt

We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response $y$,…

Methodology · Statistics 2014-03-03 Jason D Lee , Jonathan E Taylor

How to select the active variables which have significant impact on the event of interest is a very important and meaningful problem in the statistical analysis of ultrahigh-dimensional data. Sure independent screening procedure has been…

Methodology · Statistics 2023-03-28 Xuerui Li , Yanyan Liu , Yankai Peng , Jing Zhang

We propose a likelihood ratio based inferential framework for high dimensional semiparametric generalized linear models. This framework addresses a variety of challenging problems in high dimensional data analysis, including incomplete…

Machine Learning · Statistics 2015-11-24 Yang Ning , Tianqi Zhao , Han Liu

We propose an iterative variable selection scheme for high-dimensional data with binary outcomes. The scheme adopts a structured screen-and-select framework and uses non-local prior-based Bayesian model selection within the same. The…

Methodology · Statistics 2022-11-08 Nilotpal Sanyal

For the last two decades, high-dimensional data and methods have proliferated throughout the literature. Yet, the classical technique of linear regression has not lost its usefulness in applications. In fact, many high-dimensional…

Statistics Theory · Mathematics 2021-05-18 Arun Kumar Kuchibhotla , Lawrence D. Brown , Andreas Buja , Edward I. George , Linda Zhao

We consider the problem of high-dimensional non-linear variable selection for supervised learning. Our approach is based on performing linear selection among exponentially many appropriately defined positive definite kernels that…

Machine Learning · Computer Science 2009-09-08 Francis Bach

We study a marginal empirical likelihood approach in scenarios when the number of variables grows exponentially with the sample size. The marginal empirical likelihood ratios as functions of the parameters of interest are systematically…

Statistics Theory · Mathematics 2013-11-07 Jinyuan Chang , Cheng Yong Tang , Yichao Wu

The applications of traditional statistical feature selection methods to high-dimension, low sample-size data often struggle and encounter challenging problems, such as overfitting, curse of dimensionality, computational infeasibility, and…

Machine Learning · Statistics 2023-12-19 Kexuan Li , Fangfang Wang , Lingli Yang , Ruiqi Liu

High-dimensional prediction typically comprises two steps: variable selection and subsequent least-squares refitting on the selected variables. However, the standard variable selection procedures, such as the lasso, hinge on tuning…

Methodology · Statistics 2017-06-07 Didier Chételat , Johannes Lederer , Joseph Salmon

Post-selection inference is a statistical technique for determining salient variables after model or variable selection. Recently, selective inference, a kind of post-selection inference framework, has garnered the attention in the…

Methodology · Statistics 2019-06-28 Yuta Umezu , Ichiro Takeuchi

Variable selection plays an important role in high dimensional statistical modeling which nowadays appears in many areas and is key to various scientific discoveries. For problems of large scale or dimensionality $p$, estimation accuracy…

Statistics Theory · Mathematics 2008-08-27 Jianqing Fan , Jinchi Lv

Variable selection in ultrahigh-dimensional linear regression is challenging due to its high computational cost. Therefore, a screening step is usually conducted before variable selection to significantly reduce the dimension. Here we…

Methodology · Statistics 2025-04-29 Run Wang , An Nguyen , Somak Dutta , Vivekananda Roy

We demonstrate how supervised learning can be decomposed into a two-stage procedure, where (1) all model parameters are selected in an unsupervised manner, and (2) the outputs y are added to the model, without changing the parameter values.…

Machine Learning · Statistics 2026-03-30 Oskar Allerbo , Thomas B. Schön