Related papers: The Entropies
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
Ever since Claude Shannon used entropy for his "Mathematical Theory of Communication", entropy has become a buzzword in research circles with scientists applying entropy to describe any phenomena that are reminiscent of disorder. In this…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
This article is a short version of a longer article to appear in Physics Reports (cond-mat/9708200). The essential postulates of classical thermodynamics are formulated, from which the second law is deduced as the principle of increase of…
We present a quantum information theory that allows for the consistent description of quantum entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices, rather than probability…
A definition of entropy via the Kolmogorov algorithmic complexity is discussed. As examples, we show how the meanfield theory for the Ising model, and the entropy of a perfect gas can be recovered. The connection with computations are…
Microcanonical thermodynamics allows the application of statistical mechanics both to finite and even small systems and also to the largest, self-gravitating ones. However, one must reconsider the fundamental principles of statistical…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
Firstly, we calculate quantitatively decrease of entropy by the known formulas in the ordering phenomena and nucleation of thermodynamics of microstructure. They show again that a necessary condition of decrease of entropy in isolated…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
R\'enyi entropy is a one-parameter generalization of Shannon entropy, which has been used in various fields of physics. Despite its wide applicability, the physical interpretations of the R\'enyi entropy are not widely known. In this paper,…
Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…
This is a short analysis of the changes in the concept of entropy as applied to physics of the present-day and Early Universe. Of special interest is a leading role of such a notion as deformation of a physical theory. The relation to a…
In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple…
We investigate how undecidability enters into computations of classical physical systems and contributes to the increase of entropy and loss of information. In actual computation with finite bit of information capacity we accept…
A simple method for finding the entropy and redundancy of a reasonable long sample of English text by direct computer processing and from first principles according to Shannon theory is presented. As an example, results on the entropy of…
A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be…
Entropy is one of the most fundamental quantities in physics. For systems with few degrees of freedom, the value of entropy provides a powerful insight into its microscopic dynamics, such as the number, degeneracy and relative energies of…
Entropy is a fundamental concept in quantum information theory that allows to quantify entanglement and investigate its properties, for example its monogamy over multipartite systems. Here, we derive variational formulas for relative…
In systems with long-range interactions, since energy is a non-additive quantity, ensemble inequivalence can arise: it is possible that different statistical ensembles lead to different equilibrium descriptions, even in the thermodynamic…