Related papers: Is the Information Entropy the Same as the Statist…
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…
Two identities in statistical mechanics involving entropy differences (or ratios of density of states) at constant energy are derived. The first provides a nontrivial extension of the Jarzynski equality to the microcanonical ensemble [C.…
We address the generalized uncertainty principle in scenarios of successive measurements. Uncertainties are characterized by means of generalized entropies of both the R\'{e}nyi and Tsallis types. Here, specific features of measurements of…
Entropy is the distinguishing and most important concept of our efforts to understand and regularize our observations of a very large class of natural phenomena, and yet, it is one of the most contentious concepts of physics. In this…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
In the article the $mm$-entropy (an entropy of a metric measure space) introduced by C. Shannon is evaluated for an $\alpha$-stable L\'evy process. For $\alpha<1$ the double-sided estimates of the same order are obtained for process…
The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
In this paper an alternative approach to statistical mechanics based on the maximum information entropy principle (MaxEnt) is examined, specifically its close relation with the Gibbs method of ensembles. It is shown that the MaxEnt…
We study conditional linear information inequalities, i.e., linear inequalities for Shannon entropy that hold for distributions whose entropies meet some linear constraints. We prove that some conditional information inequalities cannot be…
We extend present Shannon's static statistical information theory to dynamic processes and establish a dynamic statistical information theory. We derive the nonlinear evolution equations of dynamic information density and dynamic…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
Information entropic measures such as Fisher information, Shannon entropy, Onicescu energy and Onicescu Shannon entropy of a symmetric double-well potential are calculated in both position and momentum space. Eigenvalues and eigenvectors of…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
Thermodynamic entropy is not an entirely satisfactory measure of information of a quantum state. This entropy for an unknown pure state is zero, although repeated measurements on copies of such a pure state do communicate information. In…
A definition of the thermodynamic entropy based on the time-dependent probability distribution of the macroscopic variables is developed. When a constraint in a composite system is released, the probability distribution for the new…
According to quantum mechanics, the informational content of isolated systems does not change in time. However, subadditivity of entropy seems to describe an excess of information when we look at single parts of a composite systems and…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
In the estimation theory context, we generalize the notion of Shannon's entropy power to the R\'{e}nyi-entropy setting. This not only allows to find new estimation inequalities, such as the R\'{e}nyi-entropy based De Bruijn identity,…
We investigate how undecidability enters into computations of classical physical systems and contributes to the increase of entropy and loss of information. In actual computation with finite bit of information capacity we accept…