Related papers: Model averaging: A shrinkage perspective
In this paper, we apply shrinkage strategies to estimate regression coefficients efficiently for the high-dimensional multiple regression model, where the number of samples is smaller than the number of predictors. We assume in the sparse…
Sparse additive models have attracted much attention in high-dimensional data analysis due to their flexible representation and strong interpretability. However, most existing models are limited to single-level learning under the…
In a linear regression model with random design, we consider a family of candidate models from which we want to select a `good' model for prediction out-of-sample. We fit the models using block shrinkage estimators, and we focus on the…
We propose an improved LASSO estimation technique based on Stein-rule. We shrink classical LASSO estimator using preliminary test, shrinkage, and positive-rule shrinkage principle. Simulation results have been carried out for various…
In this article we provide some nonnegative and positive estimators of the mean squared errors(MSEs) for shrinkage estimators of multivariate normal means. Proposed estimators are shown to improve on the uniformly minimum variance unbiased…
Linear shrinkage estimators of a covariance matrix --- defined by a weighted average of the sample covariance matrix and a pre-specified shrinkage target matrix --- are popular when analysing high-throughput molecular data. However, their…
In all areas of human knowledge, datasets are increasing in both size and complexity, creating the need for richer statistical models. This trend is also true for economic data, where high-dimensional and nonlinear/nonparametric inference…
Consider the problem of estimating a multivariate normal mean with a known variance matrix, which is not necessarily proportional to the identity matrix. The coordinates are shrunk directly in proportion to their variances in Efron and…
Whether deterministic or stochastic, models can be viewed as functions designed to approximate a specific quantity of interest. We introduce Minimal Empirical Variance Aggregation (MEVA), a data-driven framework that integrates predictions…
We present a linear regression method for predictions on a small data set making use of a second possibly biased data set that may be much larger. Our method fits linear regressions to the two data sets while penalizing the difference…
This paper is concerned with model averaging estimation for partially linear functional score models. These models predict a scalar response using both parametric effect of scalar predictors and non-parametric effect of a functional…
The exponential moving average (EMA) is a commonly used statistic for providing stable estimates of stochastic quantities in deep learning optimization. Recently, EMA has seen considerable use in generative models, where it is computed with…
Shrinkage methods are frequently used to improve the precision of least squares estimators of fixed effects. However, widely used shrinkage estimators guarantee improved precision only under strong distributional assumptions. I develop an…
Studies often estimate associations between an outcome and multiple variates. For example, studies of diagnostic test accuracy estimate sensitivity and specificity, and studies of predictive and prognostic factors typically estimate…
In this paper, a new ridge-type shrinkage estimator for the precision matrix has been proposed. The asymptotic optimal shrinkage coefficients and the theoretical loss were derived. Data-driven estimators for the shrinkage coefficients were…
Representation learning has been widely studied in the context of meta-learning, enabling rapid learning of new tasks through shared representations. Recent works such as MAML have explored using fine-tuning-based metrics, which measure the…
We propose the approximate Laplace approximation (ALA) to evaluate integrated likelihoods, a bottleneck in Bayesian model selection. The Laplace approximation (LA) is a popular tool that speeds up such computation and equips strong model…
Consider a problem of predicting a response variable using a set of covariates in a linear regression model. If it is \emph{a priori} known or suspected that a subset of the covariates do not significantly contribute to the overall fit of…
We consider stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use…
Data in non-Euclidean spaces are commonly encountered in many fields of Science and Engineering. For instance, in Robotics, attitude sensors capture orientation which is an element of a Lie group. In the recent past, several researchers…