Related papers: Entropy functions and determinant inequalities
The entropy of a closure operator has been recently proposed for the study of network coding and secret sharing. In this paper, we study closure operators in relation to their entropy. We first introduce four different kinds of rank…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Spectral properties of an arbitrary matrix can be characterized by the entropy of its rescaled singular values. Any quantum operation can be described by the associated dynamical matrix or by the corresponding superoperator. The entropy of…
In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…
We find a sharp combinatorial bound for the metric entropy of sets in R^n and general classes of functions. This solves two basic combinatorial conjectures on the empirical processes. 1. A class of functions satisfies the uniform Central…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
The paper deals with conditional linear information inequalities valid for entropy functions induced by discrete random variables. Specifically, the so-called conditional Ingleton inequalities are in the center of interest: these are valid…
The paper examines relationships between the conditional Shannon entropy and the expectation of $\ell_{\alpha}$-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of…
For a restricted class of potentials (harmonic+Gaussian potentials), we express the resolvent integral for the correlation functions of simple traces of powers of complex matrices of size $N$, in term of a determinant; this determinant is…
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We investigate the theoretical foundations of a recently introduced entropy-based formulation of weighted least squares for the approximation of overdetermined linear systems, motivated by robust data fitting in the presence of sparse gross…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
Quantum relative entropies are jointly convex functions of two positive definite matrices that generalize the Kullback-Leibler divergence and arise naturally in quantum information theory. In this paper, we prove self-concordance of natural…
In 1997, Z.Zhang and R.W.Yeung found the first example of a conditional information inequality in four variables that is not "Shannon-type". This linear inequality for entropies is called conditional (or constraint) since it holds only…
We present a numerical study of the application of the Shannon entropy technique to the planar restricted three-body problem in the vicinity of first-order interior mean-motion resonances with the perturber. We estimate the diffusion…