Related papers: Average Entropy Functions
In this paper, we show that the characterisation of all determinant inequalities for $n \times n$ positive definite matrices is equivalent to determining the smallest closed and convex cone containing all entropy functions induced by $n$…
Given n (discrete or continuous) random variables X_i, the (2^n-1)-dimensional vector obtained by evaluating the joint entropy of all non-empty subsets of {X_1,...,X_n} is called an entropic vector. Determining the region of entropic…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…
Characterization of entropy functions is of fundamental importance in information theory. By imposing constraints on their Shannon outer bound, i.e., the polymatroidal region, one obtains the faces of the region and entropy functions on…
Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…
The entropy region is constructed from vectors of random variables by collecting Shannon entropies of all subvectors. Its shape is studied here by means of polymatroidal constructions, notably by convolution. The closure of the region is…
We give a closed-form solution of von Neumann entropy as a function of geometric phase modulated by visibility and average distinguishability in Hilbert spaces of two and three dimensions. We show that the same type of dependence also…
Let $\cal{N}=\{1,\cdots,n\}$. The entropy function $\bf h$ of a set of $n$ discrete random variables $\{X_i:i\in\cal N\}$ is a $2^n$-dimensional vector whose entries are ${\bf{h}}({\cal{A}})\triangleq H(X_{\cal{A}}),\cal{A}\subset{\cal N}…
The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We consider the concept of generalized measure-theoretic entropy, where instead of the Shannon entropy function we consider an arbitrary concave function defined on the unit interval, vanishing in the origin. Under mild assumptions on this…
In this note, it is shown that if $f\colon\efq^n\to\efq^n$ is any function and $\bA=(A_1,..., A_n)$ is uniformly distributed over $\efq^n$, then the average over $(k_1,...,k_n)\in \efq^n$ of the Renyi (and hence, of the Shannon) entropy of…
We study the effect of the choice of embedding geometry on the entropy of random geometric graph ensembles with soft connection functions. First we show that when the connection range is small, the entropy is dependent only on the dimension…
This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…
Using convex Grothendieck fibrations, we characterize the von Neumann entropy as a functor from finite-dimensional non-commutative probability spaces and state-preserving *-homomorphisms to real numbers. Our axioms reproduce those of Baez,…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
The scattering amplitude in simple quantum graphs is a well-known process which may be highly complex. In this work, motivated by the Shannon entropy, we propose a methodology that associates to a graph a scattering entropy, which we call…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…