Related papers: Shannon Entropy Reinterpreted
Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…
The generalized stochastic Loewner evolution (SLE) driven by reversible Langevin dynamics was theoretically investigated in the context of non-equilibrium statistical mechanics. The recent study of the authors revealed that the Loewner…
In recent years, learning for neural networks can be viewed as optimization in the space of probability measures. To obtain the exponential convergence to the optimizer, the regularizing term based on Shannon entropy plays an important…
In many applications, the probability density function is subject to experimental errors. In this work the continuos dependence of a class of generalized entropies on the experimental errors is studied. This class includes the C. Shannon,…
We study the evolution of Tsallis entropy along the heat flow and establish its concavity in arbitrary dimensions. Extending prior results that were restricted to the one-dimensional setting, we prove that the Tsallis entropy is concave in…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…
Superstatistics describes statistical systems that behave like superpositions of different inverse temperatures $\beta$, so that the probability distribution is $p(\epsilon_i) \propto \int_{0}^{\infty} f(\beta) e^{-\beta \epsilon_i}d\beta$,…
Generalizing the group structure of the Euclidean space, we construct a Riemannian metric on the deformed set \ $\mathbb{R}^n_q$ \ induced by the Tsallis entropy composition property. We show that the Tsallis entropy is a "hyperbolic…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
In this paper we give an interpretation of Tsallis' nonextensive statistical mechanics based upon the information-theoretic point of view of Luzzi et al. [cond-mat/0306217; cond-mat/0306247; cond-mat/0307325], suggesting Tsallis' entropy to…
The paper deals with the generalization of both Boltzmann entropy and distribution in the light of most-probable interpretation of statistical equilibrium. The statistical analysis of the generalized entropy and distribution leads to some…
By using the maximum entropy principle with Tsallis entropy we obtain a fragment size distribution function which undergoes a transition to scaling. This distribution function reduces to those obtained by other authors using Shannon…
The aim of the present paper is to present a careful and accessible discussion of the formal aspects of Boltzmann-Gibbs and Tsallis entropies. We begin with a brief overview of Boltzmann-Gibbs entropy, highlighting its main properties and…
An axiomatic foundation regarding the entropy for complex systems is established. Missing from decades of research was the requirement that entropy must measure the uncertainty at the informational scale of the maximizing distribution,…
Isomorph theory is one of the promising theories to understand the quasi-universal relationship between thermodynamic, dynamic and structural characteristics. Based on the hidden scale invariance of the inverse power law potentials, it…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative…
We develop an entropy based framework for analyzing symmetry breaking in dynamical systems. Information transfer, which measures the directional exchange of entropy between observables, provides a quantitative early indicator of symmetry…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…