Related papers: Shannon Entropy Reinterpreted
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
We present a new approach to far-from-equilibrium statistical mechanics, based on the concept of generalized entropy, which is a microscopically-defined generalization of Onsager-Machlup functional. In the case when a set of slow…
There exists only one generalization of the classical Boltzmann-Gibbs-Shannon entropy functional to a one-parametric family of additive entropy functionals. We find analytical solution to the corresponding extension of the classical…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called…
We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…
Aim of this short note is to study Shannon's entropy power along entropic interpolations, thus generalizing Costa's concavity theorem. We shall provide two proofs of independent interest: the former by $\Gamma$-calculus, hence applicable to…
We show how Fisher's information already known particular character as the fundamental information geometric object which plays the role of a metric tensor for a statistical differential manifold, can be derived in a relatively easy manner…
We demonstrate that Shannon's information entropy and the thermodynamic entropy of Boltzmann and Gibbs are quantitatively equivalent for real condensed-matter systems. By interpreting atomic configurations as information sources, we compute…
We derive a scaling property from a fundamental nonlinear differential equation whose solution is the so-called q-exponential function. A scaling property has been believed to be given by a power function only, but actually more general…
We revisit the derivation of a formula for the $q$-generalised multinomial coefficient rooted in the $q$-deformed algebra, a foundational framework in the study of nonextensive statistics. Previous approximate expressions in the literature…
In special relativity the mathematical expressions, defining physical observables as the momentum, the energy etc, emerge as one parameter (light speed) continuous deformations of the corresponding ones of the classical physics. Here, we…
We develop variational representations for the deformed logarithmic and exponential functions and use them to obtain variational representations related to the quantum Tsallis relative entropy. We extend Golden-Thompson's trace inequality…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
Algorithmic entropy and Shannon entropy are two conceptually different information measures, as the former is based on size of programs and the later in probability distributions. However, it is known that, for any recursive probability…
In this simple article, with possible applications in theoretical and applied physics, we suggest an original way to derive the expression of Shannon's entropy from a purely variational approach,using constraints. Based on the work of Edwin…