Related papers: Generalized Information Inequalities via Submodula…
The present paper offers, in its first part, a unified approach for the derivation of families of inequalities for set functions which satisfy sub/supermodularity properties. It applies this approach for the derivation of information…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
Submodular functions are known to satisfy various forms of fractional subadditivity. This work investigates the conditions for equality to hold exactly or approximately in the fractional subadditivity of submodular functions. We establish…
Dembo, Cover, and Thomas (1991) developed an elegant information-theoretic framework for proving determinantal inequalities for positive definite matrices, which relies on the structural inequalities of differential entropy. Submodular…
In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called…
In this paper we formalize the notions of information elements and information lattices, first proposed by Shannon. Exploiting this formalization, we identify a comprehensive parallelism between information lattices and subgroup lattices.…
In this paper we have shall generalize Shearer's entropy inequality and its recent extensions by Madiman and Tetali, and shall apply projection inequalities to deduce extensions of some of the inequalities concerning sums of sets of…
With increasing volume of data being used across machine learning tasks, the capability to target specific subsets of data becomes more important. To aid in this capability, the recently proposed Submodular Mutual Information (SMI) has been…
Considering the weighted concept of majorization, Sherman obtained generalization of majorization inequality for convex functions known as Sherman's inequality. We extend Sherman's result to the class of n-strongly convex functions using…
During the last two decades, concentration inequalities have been the subject of exciting developments in various areas, including convex geometry, functional analysis, statistical physics, high-dimensional statistics, pure and applied…
An information theoretic approach to bounds in superconformal field theories is proposed. It is proved that the supersymmetric R\'enyi entropy $\bar S_\alpha$ is a monotonically decreasing function of $\alpha$ and $(\alpha-1)\bar S_\alpha$…
We propose a new way to measure inequalities such as the glass ceiling effect in attributed networks. Existing measures typically rely solely on node degree distribution or degree assortativity, but our approach goes beyond these measures…
We investigate how basic probability inequalities can be extended to an imprecise framework, where (precise) probabilities and expectations are replaced by imprecise probabilities and lower/upper previsions. We focus on inequalities giving…
We consider the problem of identifying universal low-dimensional features from high-dimensional data for inference tasks in settings involving learning. For such problems, we introduce natural notions of universality and we show a local…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
This work extends the Mond-Pecaric method to functions with multiple operators as arguments by providing arbitrarily close approximations of the original functions. Instead of using linear functions to establish lower and upper bounds for…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon…
We connect high-dimensional subset selection and submodular maximization. Our results extend the work of Das and Kempe (2011) from the setting of linear regression to arbitrary objective functions. For greedy feature selection, this…