Related papers: Asymptotically Optimal Massey-Like Inequality on G…
This paper first focuses on deriving an alternative approach for proving an extremal entropy inequality (EEI), originally presented in [11]. The proposed approach does not rely on the channel enhancement technique, and has the advantage…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
Entropic uncertainty relations place nontrivial lower bounds to the sum of Shannon information entropies for noncommuting observables. Here we obtain a novel lower bound on the entropy sum for general pairs of observables in…
In this research paper, it is proved that an approximation to Gibbs-Shannon entropy measure naturally leads to Tsallis entropy for the real parameter q =2 . Several interesting measures based on the input as well as output of a discrete…
The method of optimizing entropy is used to (i) conduct Asymptotic Hypothesis Testing and (ii) determine the particle distribution for which Entropy is maximized. This paper focuses on two related applications of Information Theory:…
For an ergodic Brownian diffusion with invariant measure $\nu$, we consider a sequence of empirical distributions ($\nu$n) n$\ge$1 associated with an approximation scheme with decreasing time step ($\gamma$n) n$\ge$1 along an adapted…
Maximum entropy method is a constructive criterion for setting up a probability distribution maximally non-committal to missing information on the basis of partial knowledge, usually stated as constrains on expectation values of some…
The correspondence between the cross-entropy method and the zero-variance approximation to simulate a rare event problem in Markov chains is shown. This leads to a sufficient condition that the cross-entropy estimator is asymptotically…
Two types of errors can occur when discriminating pairs of quantum states. Asymmetric state discrimination involves minimizing the probability of one type of error, subject to a constraint on the other. We give explicit expressions bounding…
We propose a novel method for testing isotropy of a three-dimensional distribution using Shannon entropy. We test the method on some Monte Carlo simulations of isotropic and anisotropic distributions and find that the method can effectively…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath…
In the present paper, we consider the plug-in estimator of Shannon's entropy defined on a finite alphabet which is assumed to dynamically vary as the sample size increases. The asymptotic behaviors for the plug-in estimator, such as,…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
In the present work we investigate phase correlations by recourse to the Shannon entropy. Using theoretical arguments we show that the entropy provides an accurate measure of phase correlations in any dynamical system, in particular when…
Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…
A thermodynamic framework for asymptotic inference is developed in which sample size and parameter variance define a state space. Within this description, Shannon information plays the role of entropy, and an integrating factor organizes…
Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…