Related papers: Estimation of entropy-type integral functionals
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
We present two new estimators for estimating the entropy of absolutely continuous random variables. Some properties of them are considered, specifically consistency of the first is proved. The introduced estimators are compared with the…
In statistical analysis, quantifying uncertainties through measures such as entropy, extropy, varentropy, and varextropy is of fundamental importance for understanding distribution functions. This paper investigates several properties of…
This paper concerns the estimation of sums of functions of observable and unobservable variables. Lower bounds for the asymptotic variance and a convolution theorem are derived in general finite- and infinite-dimensional models. An explicit…
In this monograph, we prove an asymptotic approximation for integrals of probability densities over sets in finite dimensional euclidean space, which are far away from the origin (asymptotic sets). We use this approximation to investigate…
Irregular functional data in which densely sampled curves are observed over different ranges pose a challenge for modeling and inference, and sensitivity to outlier curves is a concern in applications. Motivated by applications in…
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic…
We analyze a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
We consider the problem of estimating smooth integrated functionals of a monotone nonincreasing density $f$ on $[0,\infty)$ using the nonparametric maximum likelihood based plug-in estimator. We find the exact asymptotic distribution of…
A new sharp inequality featuring the differential R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross-entropy of a pair of probability density functions is established. The equality is reached when one of the probability density…
Asymptotic statistical theory for estimating functions is reviewed in a generality suitable for stochastic processes. Conditions concerning existence of a consistent estimator, uniqueness, rate of convergence, and the asymptotic…
A unified formulation of the density functional theory is constructed on the foundations of entropic inference in both the classical and the quantum regimes. The theory is introduced as an application of entropic inference for inhomogeneous…
We present two classes of improved estimators for mutual information $M(X,Y)$, from samples of random points distributed according to some joint probability density $\mu(x,y)$. In contrast to conventional estimators based on binnings, they…
The universal typical-signal estimators of entropy and cross entropy based on the asymptotics of recurrence and waiting times play an important role in information theory. Building on their construction, we introduce and study universal…
When the study variable is functional and storage capacities are limited or transmission costs are high, selecting with survey sampling techniques a small fraction of the observations is an interesting alternative to signal compression…
A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-R\'eyni-Tsallis entropies estimates. From there almost sure rates of…
In this paper, we study the asymptotic distribution of some U-statistics whose entries are functions of empirical moments computed from non-overlapping consecutive blocks of an underlying weakly dependent process. The length of these blocks…
Given a sample of independent and identically distributed random variables, a novel nonparametric maximum entropy method is presented to estimate the underlying continuous univariate probability density function (pdf). Estimates are found…
In this paper, we derive the joint asymptotic distributions of functions of quantile estimators (the non-parametric sample quantile and the parametric location-scale quantile estimator) with functions of measure of dispersion estimators…