Related papers: Extropy: Complementary Dual of Entropy
We propose an entropy-based information measure, namely the Discounted Least Information Theory of Entropy (DLITE), which not only exhibits important characteristics expected as an information measure but also satisfies conditions of a…
The Boltzmann--Gibbs entropy is a functional on the space of probability measures. When a state space is countable, one characterization of the Boltzmann--Gibbs entropy is given by the Shannon--Khinchin axioms, which consist of continuity,…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
An amended MaxEnt formulation for systems displaced from the conventional MaxEnt equilibrium is proposed. This formulation involves the minimization of the Kullback-Leibler divergence to a reference $Q$ (or maximization of Shannon…
In this simple article, with possible applications in theoretical and applied physics, we suggest an original way to derive the expression of Shannon's entropy from a purely variational approach,using constraints. Based on the work of Edwin…
We consider the uncertainty between two pairs of local projective measurements performed on a multipartite system. We show that the optimal bound in any linear uncertainty relation, formulated in terms of the Shannon entropy, is additive.…
This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
In this research paper, it is proved that an approximation to Gibbs-Shannon entropy measure naturally leads to Tsallis entropy for the real parameter q =2 . Several interesting measures based on the input as well as output of a discrete…
The fractional order generalization of Shannon entropy proposed by Ubriaco has been studied for discrete distributions. In the current paper, we conduct a detailed study of the continuous analogue of this entropy termed as fractional…
The diversity of the symbols of the information source is calculated following the definition that entropy is the information loss and following a new entropy-symbol similarity relation after the rejection of the Gibbs paradox statement.…
Shannon and Renyi entropies are quantitative measures of uncertainty in a data set. They are developed by Renyi in the context of entropy theory. These measures have been studied in the case of the multivariate t-distributions. We extend…
Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…
The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
Though Shannon entropy of a probability measure $P$, defined as $- \int_{X} \frac{\ud P}{\ud \mu} \ln \frac{\ud P}{\ud\mu} \ud \mu$ on a measure space $(X, \mathfrak{M},\mu)$, does not qualify itself as an information measure (it is not a…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
Existing research on negations primarily focuses on entropy and extropy. Recently, new functions such as varentropy and varextropy have been developed, which can be considered as extensions of entropy and extropy. However, the impact of…