Related papers: Extropy: Complementary Dual of Entropy
We share a small connection between information theory, algebra, and topology - namely, a correspondence between Shannon entropy and derivations of the operad of topological simplices. We begin with a brief review of operads and their…
In this paper we derive the maximum entropy characteristics of a particular rank order distribution, namely the discrete generalized beta distribution, which has recently been observed to be extremely useful in modelling many several…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
We introduce a simple geometrical construction similar to covariant holographic entanglement entropy but with the addition of a new term proportional to boundary region volume. This new procedure has properties strongly resembling classical…
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two…
Inferring and comparing complex, multivariable probability density functions is fundamental to problems in several fields, including probabilistic learning, network theory, and data analysis. Classification and prediction are the two faces…
Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann- Gibbs form of the entropy ensures that…
The fundamentals of the Maximum Entropy principle as a rule for assigning and updating probabilities are revisited. The Shannon-Jaynes relative entropy is vindicated as the optimal criterion for use with an updating rule. A constructive…
Expanding Thurston maps were introduced by M. Bonk and D. Meyer with motivation from complex dynamics and Cannon's conjecture from geometric group theory via Sullivan's dictionary. In this paper, we show that the entropy map of an expanding…
We show the equivalence of three properties for an infinitely divisible distribution: the subexponentiality of the density, the subexponentiality of the density of its L\'evy measure and the tail equivalence between the density and its…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
The aim of this paper is to investigate the q -> 1/q duality in an information-entropy theory of all q-generalized entropy functionals (Tsallis, Renyi and Sharma-Mittal measures) in the light of a representation based on generalized…
A measure is derived to quantify directed information transfer between pairs of vertices in a weighted network, over paths of a specified maximal length. Our approach employs a general, probabilistic model of network traffic, from which the…
Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…
We analyze a contrasting dynamical behavior of Gibbs-Shannon and conditional Kullback-Leibler entropies, induced by time-evolution of continuous probability distributions. The question of predominantly purpose-dependent entropy definition…
The standard formulation of thermostatistics, being based on the Boltzmann-Gibbs distribution and logarithmic Shannon entropy, describes idealized uncorrelated systems with extensive energies and short-range interactions. In this letter, we…
The asymptotic convergence of probability density function (pdf) and convergence of differential entropy are examined for the non-stationary processes that follow the maximum entropy principle (MaxEnt) and maximum entropy production…
Entropy production is the crucial quantity characterizing irreversible phenomena and the second law of thermodynamics. Yet, a ubiquitous definition eludes consensus. Given that entropy production arises from incomplete access to…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…