Related papers: A note on Shannon entropy
We give a very simple proof of a strengthened version of Chernoff's Inequality. We derive the same conclusion from much weaker assumptions.
We present a relative form of the Toponogov comparison theorem.
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
Entropies of mixing can be derived directly from the parent distributions of extreme value theory. They correspond to pseudo-additive entropies in the case of Pareto and power function distributions, while to the Shannon entropy in the case…
Recent theoretical developments revived the interest in charged particle multiplicities and their wide-spread parametrization, the negative binomial distribution (NBD). The central observable of the studies is the Shannon entropy of the…
We review subbadditivity properties of Shannon entropy, in particular, from the Shearer's inequality we derive the "infimum rule" for actions of amenable groups. We briefly discuss applicability of the "infimum formula" to actions of other…
We normalize the combinatorial Laplacian of a graph by the degree sum, look at its eigenvalues as a probability distribution and then study its Shannon entropy. Equivalently, we represent a graph with a quantum mechanical state and study…
This is an expository article on relating the Chebotarev Density Theorem to the Bateman-Horn constant.
Product probability property, known in the literature as statistical independence, is examined first. Then generalized entropies are introduced, all of which give generalizations to Shannon entropy. It is shown that the nature of the…
Entropy increase is fundamentally related to the breaking of time-reversal symmetry. By adding the 'extra dimension' associated with thermodynamic forces, we extend that discrete symmetry to a continuous symmetry for the dynamical…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
It is known that many classical inequalities linked to convolutions can be obtained by looking at the monotonicity in time of convolutions of powers of solutions to the heat equation, provided that both the exponents and the coefficients of…
The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we…
In this note we give a detailed proof of a theorem of Aubin.
Linear, irreversible thermodynamics predicts that the entropy production rate can become negative. We demonstrate this prediction for metals under AC-driving whose conductivity is well-described by the Drude-Sommerfeld model. We then show…
We prove characterization theorems for relative entropy (also known as Kullback-Leibler divergence), q-logarithmic entropy (also known as Tsallis entropy), and q-logarithmic relative entropy. All three have been characterized axiomatically…
Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…
We generalize the Clausius (in)equality to overdamped mesoscopic and macroscopic diffusions in the presence of nonconservative forces. In contrast to previous frameworks, we use a decomposition scheme for heat which is based on an exact…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
We review a new form of entropy suggested by us, with origin in mixing of states of systems due to interactions and deformations of phase cells. It is demonstrated that this nonextensive form also leads to asymmetric maximal entropy…