Related papers: On Improved Loss Estimation for Shrinkage Estimato…

This work addresses large dimensional covariance matrix estimation with unknown mean. The empirical covariance estimator fails when dimension and number of samples are proportional and tend to infinity, settings known as Kolmogorov…

A generic out-of-sample error estimate is proposed for robust $M$-estimators regularized with a convex penalty in high-dimensional linear regression where $(X,y)$ is observed and $p,n$ are of the same order. If $\psi$ is the derivative of…

When developing risk prediction models, shrinkage methods are recommended, especially when the sample size is limited. Several earlier studies have shown that the shrinkage of model coefficients can reduce overfitting of the prediction…

In this paper we derive the optimal linear shrinkage estimator for the high-dimensional mean vector using random matrix theory. The results are obtained under the assumption that both the dimension $p$ and the sample size $n$ tend to…

In this work we study loss functions for learning and evaluating probability distributions over large discrete domains. Unlike classification or regression where a wide variety of loss functions are used, in the distribution learning and…

The logistic loss function is often advocated in machine learning and statistics as a smooth and strictly convex surrogate for the 0-1 loss. In this paper we investigate the question of whether these smoothness and convexity properties make…

We propose a novel sensitivity analysis framework for linear estimators with identification failures that can be viewed as seeing the wrong outcome distribution. Our approach measures the degree of identification failure through the change…

Let $(X_i)_{i=1,...,n}$ be a possibly nonstationary sequence such that $\mathscr{L}(X_i)=P_n$ if $i\leq n\theta$ and $\mathscr{L}(X_i)=Q_n$ if $i>n\theta$, where $0<\theta <1$ is the location of the change-point to be estimated. We…

Most zeroth-order optimization algorithms mimic a first-order algorithm but replace the gradient of the objective function with some gradient estimator that can be computed from a small number of function evaluations. This estimator is…

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent…

We provide a general constrained risk inequality that applies to arbitrary non-decreasing losses, extending a result of Brown and Low [Ann. Stat. 1996]. Given two distributions $P_0$ and $P_1$, we find a lower bound for the risk of…

Three types of explicit estimators are proposed here to estimate the loss rates of the links in a network of the tree topology. All of them are derived by the maximum likelihood principle and proved to be either asymptotic unbiased or…

Central moments and cumulants are often employed to characterize the distribution of data. The skewness and kurtosis are particularly useful for the detection of outliers, the assessment of departures from normally distributed data,…

A fundamental challenge in machine learning is the choice of a loss as it characterizes our learning task, is minimized in the training phase, and serves as an evaluation criterion for estimators. Proper losses are commonly chosen, ensuring…

This paper focuses on Bayesian shrinkage for covariance matrix estimation. We examine posterior properties and frequentist risks of Bayesian estimators based on new hierarchical inverse-Wishart priors. More precisely, we give the existence…

The likelihood functions for discretely observed nonlinear continuous-time models based on stochastic differential equations are not available except for a few cases. Various parameter estimation techniques have been proposed, each with…

The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In…

An approach to inference for relative sparsity was developed in prior work, and an adaptive lasso asymptotic normality theorem was given there, but this theorem was not fully used when estimating the variance of the policy coefficients.…

To address model uncertainty under flexible loss functions in prediction problems, we propose a model averaging method that accommodates various loss functions, including asymmetric linear and quadratic loss functions, as well as many other…

This paper is concerned with inference on the regression function of a high-dimensional linear model when outcomes are missing at random. We propose an estimator which combines a Lasso pilot estimate of the regression function with a bias…