English
Related papers

Related papers: Empirical estimation of entropy functionals with c…

200 papers

Many statistical procedures, including goodness-of-fit tests and methods for independent component analysis, rely critically on the estimation of the entropy of a distribution. In this paper, we seek entropy estimators that are efficient…

Statistics Theory · Mathematics 2017-06-23 Thomas B. Berrett , Richard J. Samworth , Ming Yuan

This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…

Methodology · Statistics 2020-01-14 Amadou Diadie Ba , Gane Samb Lo , Cheikh Tidiane Seck

Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…

Information Theory · Computer Science 2023-08-22 Ziqiao Ao , Jinglai Li

In the present paper, we consider the plug-in estimator of Shannon's entropy defined on a finite alphabet which is assumed to dynamically vary as the sample size increases. The asymptotic behaviors for the plug-in estimator, such as,…

Probability · Mathematics 2024-10-01 Zhenhong Yu , Yu Miao

The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi…

Statistics Theory · Mathematics 2013-03-08 David Källberg , Nikolaj Leonenko , Oleg Seleznjev

Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…

Statistics Theory · Mathematics 2011-03-28 David Källberg , Nikolaj Leonenko , Oleg Seleznjev

Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…

Information Theory · Computer Science 2020-12-22 Yuval Shalev , Amichai Painsky , Irad Ben-Gal

The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…

Information Theory · Computer Science 2021-09-01 Leonardo Ricci , Alessio Perinelli , Michele Castelluzzo

Currently, the high-precision estimation of nonlinear parameters such as Gini indices, low-income proportions or other measures of inequality is particularly crucial. In the present paper, we propose a general class of estimators for such…

Methodology · Statistics 2014-07-01 Camelia Goga , Anne Ruiz-Gazen

Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of…

Statistics Theory · Mathematics 2016-07-22 Shashank Singh , Barnabás Póczos

Partly motivated by entropy-estimation problems in neuroscience, we present a detailed and extensive comparison between some of the most popular and effective entropy estimation methods used in practice: The plug-in method, four different…

Information Theory · Computer Science 2015-05-13 Y. Gao , I. Kontoyiannis , E. Bienenstock

The best-known and most commonly used distribution-property estimation technique uses a plug-in estimator, with empirical frequency replacing the underlying distribution. We present novel linear-time-computable estimators that significantly…

Statistics Theory · Mathematics 2019-03-06 Yi Hao , Alon Orlitsky

Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n…

Information Theory · Computer Science 2016-02-19 Yihong Wu , Pengkun Yang

In this article, we construct semiparametrically efficient estimators of linear functionals of a probability measure in the presence of side information using an easy empirical likelihood approach. We use estimated constraint functions and…

Methodology · Statistics 2023-03-01 Shan Wang , Hanxiang Peng

In this paper, a nonparametric maximum likelihood (ML) estimator for band-limited (BL) probability density functions (pdfs) is proposed. The BLML estimator is consistent and computationally efficient. To compute the BLML estimator, three…

Machine Learning · Statistics 2015-06-30 Rahul Agarwal , Zhe Chen , Sridevi V. Sarma

The min-entropy is a widely used metric to quantify the randomness of generated random numbers in cryptographic applications; it measures the difficulty of guessing the most likely output. An important min-entropy estimator is the…

Cryptography and Security · Computer Science 2021-04-05 Yongjune Kim , Cyril Guyot , Young-Sik Kim

Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying…

Information Theory · Computer Science 2017-05-19 Alan Wisler , Kevin Moon , Visar Berisha

Given an i.i.d. sample $X_1,...,X_n$ with common bounded density $f_0$ belonging to a Sobolev space of order $\alpha$ over the real line, estimation of the quadratic functional $\int_{\mathbb{R}}f_0^2(x) \mathrm{d}x$ is considered. It is…

Statistics Theory · Mathematics 2008-12-18 Evarist Giné , Richard Nickl

In the multiple testing context, a challenging problem is the estimation of the proportion $\pi_0$ of true-null hypotheses. A large number of estimators of this quantity rely on identifiability assumptions that either appear to be violated…

Statistics Theory · Mathematics 2008-12-18 Alain Celisse , Stéphane Robin

For studies in reliability, biometry, and survival analysis, the length-biased distribution is often well-suited for certain natural sampling plans. In this paper, we study the strong uniform consistency of two nonparametric estimators for…

Methodology · Statistics 2025-09-22 Vaishnavi Pavithradas , Rajesh G