相关论文: Axiomatic Origins of Mathematical Entropy: Grading…
The concept of Relative Divergence of one Grading Function from another is extended from totally ordered chains to power sets of finite event spaces. Shannon Entropy concept is extended to normalized grading functions on such power sets.…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
The concept of Shannon Entropy for probability distributions and associated Maximum Entropy Principle are extended here to the concepts of Relative Divergence of one Grading Function from another and Maximum Relative Divergence Principle…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
We share a small connection between information theory, algebra, and topology - namely, a correspondence between Shannon entropy and derivations of the operad of topological simplices. We begin with a brief review of operads and their…
Entropies of mixing can be derived directly from the parent distributions of extreme value theory. They correspond to pseudo-additive entropies in the case of Pareto and power function distributions, while to the Shannon entropy in the case…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
Why are most empirical networks, with the prominent exception of social ones, generically degree-degree anticorrelated, i.e. disassortative? With a view to answering this long-standing question, we define a general class of degree-degree…
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
The Shannon entropy is used as a basis for applying different lemmas and conjectures concerning the set of gaps between prime numbers G_p , thus estimating several measures of it. The same procedures are applied to artificially created…
Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
A number of inequalities for the weighted entropies is proposed, mirroring properties of a standard (Shannon) entropy and related quantities.
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…