Related papers: Conditional Shannon, R\'eyni, and Tsallis entropie…
We study the estimation of Tsallis entropy of a finite number of independent populations, each following an exponential distribution with the same scale parameter and distinct location parameters for $q>0$. We derive a Stein-type improved…
In the present paper, we consider the plug-in estimator of Shannon's entropy defined on a finite alphabet which is assumed to dynamically vary as the sample size increases. The asymptotic behaviors for the plug-in estimator, such as,…
We consider the meta-equilibrium state of a composite system made up of independent subsystems satisfying the additive form of external constraints, as recently discussed by Abe [Phys. Rev. E {\bf 63}, 061105 (2001)]. We derive the additive…
A general investigation is made into the intrinsic Riemannian geometry for complex systems, from the perspective of statistical mechanics. The entropic formulation of statistical mechanics is the ingredient which enables a connection…
A definition of the nonadditive (nonextensive) conditional entropy indexed by q is presented. Based on the composition law in terms of it, the Shannon-Khinchin axioms are generalized and the uniqueness theorem is established for the Tsallis…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
We consider the uncertainty between two pairs of local projective measurements performed on a multipartite system. We show that the optimal bound in any linear uncertainty relation, formulated in terms of the Shannon entropy, is additive.…
The Tsallis entropy, which is a generalization of the Boltzmann-Gibbs entropy, plays a central role in nonextensive statistical mechanics of complex systems. A lot of efforts have recently been made on establishing a dynamical foundation…
We show that the non-additivity relation of the Tsallis entropies in nonextensive statistical mechanics has a simple physical interpretation for systems with fluctuating temperature or fluctuating energy dissipation rate. We also show that…
The necessary conditions (NC) that reconcile canonical probability distributions obtained from the q-maximum entropy principle, subjected to both i) the additive duality of generalized statistics and ii) normal averages expectations with…
We show that if one uses the invariant form of the Boltzmann-Shannon continuous entropy, it is possible to obtain the generalized Pareto-Tsallis density function, using an appropriate "prior" measure m_{q}(x) and a "Boltzman constraint"…
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
In this paper we give an interpretation of Tsallis' nonextensive statistical mechanics based upon the information-theoretic point of view of Luzzi et al. [cond-mat/0306217; cond-mat/0306247; cond-mat/0307325], suggesting Tsallis' entropy to…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
We study nonparametric estimators of conditional Kendall's tau, a measure of concordance between two random variables given some covariates. We prove non-asymptotic bounds with explicit constants, that hold with high probabilities. We…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…