Related papers: Shannon Entropy Reinterpreted
Thermodynamic entropy is determined by a heat measurement through the Clausius equality. The entropy then formalizes a fundamental limitation of operations by the second law of thermodynamics. The entropy is also expressed as the Shannon…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Entropic measures provide analytic tools to help us understand correlation in quantum systems. In our previous work, we calculated linear entropy and von Neumann entropy as entanglement measures for the ground state and lower lying excited…
The translated logarithmic Lambert function is defined and basic analytic properties of the function are obtained including the derivative, integral, Taylor series expansion, real branches and asymptotic approximation of the function.…
We prove the full equivalence between Assembly Theory (AT) and Shannon Entropy via a method based upon the principles of statistical compression renamed `assembly index' that belongs to the LZ family of popular compression algorithms (ZIP,…
We start with reviewing the origin of the idea that entropy and the Second Law are associated with the Arrow of Time. We then introduced a new definition of entropy based on Shannons Measure of Information, SMI. The SMI may be defined on…
We study the properties of Tsallis entropy and Shannon entropy from the point of view of algorithmic randomness. In algorithmic information theory, there are two equivalent ways to define the program-size complexity K(s) of a given finite…
The notion of entropy is ubiquitous both in natural and social sciences. In the last two decades, a considerable effort has been devoted to the study of new entropic forms, which generalize the standard Boltzmann-Gibbs (BG) entropy and are…
In this paper, we attempt to derive the expression of ensemble average internal energy in long-range interaction complex system. Further, the Shannon entropy hypothesis is used to derive the probability distribution function of energy. It…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
Alternative definitions are given of basic concepts of generalized thermostatistics. In particular, generalizations of Shannon's entropy, of the Boltzmann-Gibbs distribution, and of relative entropy are considered. Particular choices made…
Though Shannon entropy of a probability measure $P$, defined as $- \int_{X} \frac{\ud P}{\ud \mu} \ln \frac{\ud P}{\ud\mu} \ud \mu$ on a measure space $(X, \mathfrak{M},\mu)$, does not qualify itself as an information measure (it is not a…
The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the…
A $q$-Gaussian measure is a generalization of a Gaussian measure. This generalization is obtained by replacing the exponential function with the power function of exponent $1/(1-q)$ ($q\neq 1$). The limit case $q=1$ recovers a Gaussian…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…
We consider the problem of defining free energy and other thermodynamic functions when the entropy is given as a general function of the probablity distribution, including that for non extensive forms. We find that the free energy, which is…
We address the generalized uncertainty principle in scenarios of successive measurements. Uncertainties are characterized by means of generalized entropies of both the R\'{e}nyi and Tsallis types. Here, specific features of measurements of…
The pathway model of Mathai (2005) is shown to be inferable from the maximization of a certain generalized entropy measure. This entropy is a variant of the generalized entropy of order 'alpha', considered in Mathai and Rathie (1975), and…