Related papers: On the effectiveness of a binless entropy estimato…

We present a general non-parametric statistical inference theory for integrals of quantiles without assuming any specific sampling design or dependence structure. Technical considerations are accompanied by examples and discussions,…

This paper presents a simple method for carrying out inference in a wide variety of possibly nonlinear IV models under weak assumptions. The method is non-asymptotic in the sense that it provides a finite sample bound on the difference…

The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the…

Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…

We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…

We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length.

We analyze a contrasting dynamical behavior of Gibbs-Shannon and conditional Kullback-Leibler entropies, induced by time-evolution of continuous probability distributions. The question of predominantly purpose-dependent entropy definition…

We extend Bobkov and Chistyakov's (2015) upper bounds on concentration functions of sums of independent random variables to a multivariate entropic setting. The approach is based on pointwise estimates on densities of sums of independent…

A well-known result across information theory, machine learning, and statistical physics shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This is used for…

We study asymptotic behaviors of Bayes type estimators and give sufficient conditions to obtain asymptotic limit distribution of estimation error. We assume polynomial type large deviation inequalities and prove asymptotic equivalence of…

The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…

We propose an estimator of the Hilbert-Schmidt Independence Criterion obtained from an appropriate modification of the usual estimator. We then get asymptotic normality of this estimator both under independence hypothesis and under the…

Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

A compound Poisson process whose parameters are all unknown is observed at finitely many equispaced times. Nonparametric estimators of the jump and L\'evy distributions are proposed and functional central limit theorems using the uniform…

Recently, in [P. Jizba and J. Korbel, Physica A 444, 2016, 808-827], four generalized Shannon-Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we…

The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…

We provide a unified approach to a method of estimation of the regression parameter in balanced linear models with a structured covariance matrix that combines a high breakdown point and bounded influence with high asymptotic efficiency at…

We show that the limiting variance of a sequence of estimators for a structured covariance matrix has a general form that appears as the variance of a scaled projection of a random matrix that is of radial type and a similar result is…

This article is dedicated to the estimation of the regression function when the explanatory variable is a weakly dependent process whose correlation coefficient exhibits exponential decay and has a known bounded density function. The…