Related papers: Shrinkage Estimators Dominating Some Naive Estimat…

Consider two independent exponential populations having different unknown location parameters and a common unknown scale parameter. Call the population associated with the larger location parameter as the "best" population and the…

We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…

Let us consider $k ~(\ge 2)$ independent populations $\Pi_1, \ldots,\Pi_k$, where $\Pi_i$ follows exponential distribution with hazard rate ${\sigma_i},$ ($i = 1,\ldots,k$). Suppose $Y_{i1},\ldots, Y_{in}$ be a random sample of size $n$…

This review traces the evolution of theory that started when Charles Stein in 1955 [In Proc. 3rd Berkeley Sympos. Math. Statist. Probab. I (1956) 197--206, Univ. California Press] showed that using each separate sample mean from $k\ge3$…

We study shrinkage estimation of the mean parameters of a class of multivariate distributions for which the diagonal entries of the corresponding covariance matrix are certain quadratic functions of the mean parameter. This class of…

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…

We consider estimating the Shannon entropy of a discrete distribution $P$ from $n$ i.i.d. samples. Recently, Jiao, Venkat, Han, and Weissman, and Wu and Yang constructed approximation theoretic estimators that achieve the minimax $L_2$…

We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the Spiked Covariance model and use of orthogonally invariant…

One of the major challenges in multivariate analysis is the estimation of population covariance matrix from sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results…

Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…

Let $X$ be a random vector with distribution $P_{\theta}$ where $\theta$ is an unknown parameter. When estimating $\theta$ by some estimator $\varphi(X)$ under a loss function $L(\theta,\varphi)$, classical decision theory advocates that…

We revisit the well-studied problem of estimating the Shannon entropy of a probability distribution, now given access to a probability-revealing conditional sampling oracle. In this model, the oracle takes as input the representation of a…

In a linear regression model with random design, we consider a family of candidate models from which we want to select a `good' model for prediction out-of-sample. We fit the models using block shrinkage estimators, and we focus on the…

We study a seemingly unexpected and relatively less understood overfitting aspect of a fundamental tool in sparse linear modeling - best subset selection, which minimizes the residual sum of squares subject to a constraint on the number of…

In this research work, a total of 45 different estimators of the Shannon differential entropy were reviewed. The estimators were mainly based on three classes, namely: window size spacings, kernel density estimation (KDE) and k-nearest…

This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…

Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…

We tackle covariance estimation in low-sample scenarios, employing a structured covariance matrix with shrinkage methods. These involve convexly combining a low-bias/high-variance empirical estimate with a biased regularization estimator,…

Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n…

This paper is speculated to propose a class of shrinkage estimators for shape parameter beta in failure censored samples from two-parameter Weibull distribution when some 'apriori' or guessed interval containing the parameter beta is…