Related papers: A note on entropy estimation
We propose the use of a simple intuitive principle for measuring algorithmic classification bias: the significance of the differences in a classifier's error rates across the various demographics is inversely commensurate with the sample…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
Several new estimation methods have been recently proposed for the linear regression model with observation error in the design. Different assumptions on the data generating process have motivated different estimators and analysis. In…
Regardless of studies and debates over a century, the statistical origin of the second law of thermodynamics still remains illusive. One essential obstacle is the lack of a proper theoretical formalism for non-equilibrium entropy. Here I…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
The Horvitz-Thompson (HT) estimator is widely used in survey sampling. However, the variance of the HT estimator becomes large when the inclusion probabilities are highly heterogeneous. To overcome this shortcoming, in this paper, a…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
In this paper we have adapted Bahl and Tuteja (1991) estimator in systematic sampling using auxiliary information. Using Bedi (1996) transformation an improved estimator is also proposed under systematic sampling. The expressions of bias…
This contribution reviews critically the existing entropy measures for probabilistic hesitant fuzzy sets (PHFSs), and demonstrates that these entropy measures fail to effectively distinguish a variety of different PHFSs in some cases. In…
Two identities in statistical mechanics involving entropy differences (or ratios of density of states) at constant energy are derived. The first provides a nontrivial extension of the Jarzynski equality to the microcanonical ensemble [C.…
In many important statistical applications, the number of variables or parameters $p$ is much larger than the number of observations $n$. Suppose then that we have observations $y=X\beta+z$, where $\beta\in\mathbf{R}^p$ is a parameter…
The entropy is one of the most applicable uncertainty measures in many statistical and en- gineering problems. In statistical literature, the entropy is used in calculation of the Kullback- Leibler (KL) information which is a powerful mean…
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The…
In order to study as a whole a wide part of entropy measures, we introduce a two-parameter non-extensive entropic form with respect to the $h$-derivative, which generalizes the conventional Newton--Leibniz calculus. This new entropy,…
In a recent Letter [1] a framework for estimating entropy was introduced and applied to one-dimensional and two-dimensional systems. In this Comment we show that the method is not well suited for estimating entropy in bidimensional systems…
When two independent analog signals, X and Y are added together giving Z=X+Y, the entropy of Z, H(Z), is not a simple function of the entropies H(X) and H(Y), but rather depends on the details of X and Y's distributions. Nevertheless, the…
Nested sampling has emerged as a valuable tool for Bayesian analysis, in particular for determining the Bayesian evidence. The method is based on a specific type of random sampling of the likelihood function and prior volume of the…
We study in details the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko for a large class of densities on $\mathbb{R}^d$. We then use the work of Bickel and Breiman to prove a central limit theorem in…
Two-stage sampling designs are commonly used for household and health surveys. To produce reliable estimators with assorted confidence intervals, some basic statistical properties like consistency and asymptotic normality of the…