Related papers: Spectral Conditions for the Ingleton Inequality
We formulate and analyze the compound information bottleneck programming. In this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow \mathsf{Z} $ is assumed with fixed marginal distributions $\mathsf{P}_{\mathsf{X}}$…
Cheeger-type inequalities in which the decomposability of a graph and the spectral gap of its Laplacian mutually control each other play an important role in graph theory and network analysis, in particular in the context of expander…
It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the…
This paper is concerned with certain invariant random processes (called factors of IID) on infinite trees. Given such a process, one can assign entropies to different finite subgraphs of the tree. There are linear inequalities between these…
In this paper, we establish some general forms of the law of the iterated logarithm for independent random variables in a sub-linear expectation space, where the random variables are not necessarily identically distributed. Exponential…
In this paper, we generalize the log-Sobolev inequalities to R\'enyi--Sobolev inequalities by replacing the entropy with the two-parameter entropy, which is a generalized version of entropy and closely related to R\'enyi divergences. We…
We prove the following exponential inequality: Let $n\geq 1$ and let $X_1,...,X_n$ be $n$ independent identically distributed symmetric real-valued random variables. For any $x,y>0$, we have \[\mathbb{P}\big({X_1+...+X_n}\geq x,\,…
This paper provides a finite sample bound for the error term in the Edgeworth expansion for a sum of independent, potentially discrete, nonlattice random vectors, using a uniform-in-$P$ version of the weaker Cram\'{e}r condition in Angst…
The discrete Cheeger inequality, due to Alon and Milman (J. Comb. Theory Series B 1985), is an indispensable tool for converting the combinatorial condition of graph expansion to an algebraic condition on the eigenvalues of the graph…
Although G\"odel's incompleteness theorem made mathematician recognize that no axiomatic system could completely prove its correctness and that there is an eternal hole between our knowledge and the world, physicists so far continue to work…
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…
The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset. Classical information criteria typically consider the large-data limit,…
Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number…
The quadratic decaying property of the information rate function states that given a fixed conditional distribution $p_{\mathsf{Y}|\mathsf{X}}$, the mutual information between the (finite) discrete random variables $\mathsf{X}$ and…
We introduce an alternative to the method of matched asymptotic expansions. In the "traditional" implementation, approximate solutions, valid in different (but overlapping) regions are matched by using "intermediate" variables. Here we…
The fields of quantum non-locality in physics, and causal discovery in machine learning, both face the problem of deciding whether observed data is compatible with a presumed causal relationship between the variables (for example a local…
The Leggett inequality is a constraint on the bipartite correlation that admits certain types of non-localities. Existing tests mainly focused on the electromagnetic systems where measurement apparatus are assumed to be projective and…
While the linear Pearson correlation coefficient represents a well-established normalized measure to quantify the interrelation of two stochastic variables $X$ and $Y$, it fails for multidimensional variables such as Cartesian coordinates.…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…