相关论文: Mutual information matrices are not always positiv…
A new finite form of de Finetti's representation theorem is established using elementary information-theoretic tools. The distribution of the first $k$ random variables in an exchangeable vector of $n\geq k$ random variables is close to a…
We study the problem of using i.i.d. samples from an unknown multivariate probability distribution $p$ to estimate the mutual information of $p$. This problem has recently received attention in two settings: (1) where $p$ is assumed to be…
Random matrices arise in many mathematical contexts, and it is natural to ask about the properties that such matrices satisfy. If we choose a matrix with integer entries at random, for example, what is the probability that it will have a…
The information that two random variables $Y$, $Z$ contain about a third random variable $X$ can have aspects of shared information (contained in both $Y$ and $Z$), of complementary information (only available from $(Y,Z)$ together) and of…
Given a partition $\{I_1,\ldots,I_k\}$ of $\{1,\ldots,n\}$, let $(X_1,\ldots,X_n)$ be random vector with each $X_i$ taking values in an arbitrary measurable space $(S,\mathscr{S})$ such that their joint law is invariant under finite…
We examine a definition of the mutual information of two reals proposed by Levin. The mutual information is I(X:Y)=log(sum(2^{K(s)-K^X(s)+K(t)-K^Y(t)-K(s,t)}, (s,t) pairs of finite binary strings), where K is the prefix-free Kolmogorov…
In the probabilistic approach to uncertainty management the input knowledge is usually represented by means of some probability distributions. In this paper we assume that the input knowledge is given by two discrete conditional probability…
The uncertainty relation for continuous variables due to Byalinicki-Birula and Mycielski expresses the complementarity between two $n$-uples of canonically conjugate variables $(x_1,x_2,\cdots x_n)$ and $(p_1,p_2,\cdots p_n)$ in terms of…
$H$-mutual information ($H$-MI) is a wide class of information leakage measures, where $H=(\eta, F)$ is a pair of monotonically increasing function $\eta$ and a concave function $F$, which is a generalization of Shannon entropy. $H$-MI is…
Mutual information is widely used in artificial intelligence, in a descriptive way, to measure the stochastic dependence of discrete random variables. In order to address questions such as the reliability of the empirical value, one must…
Mutual information is widely used in artificial intelligence, in a descriptive way, to measure the stochastic dependence of discrete random variables. In order to address questions such as the reliability of the empirical value, one must…
We propose a novel estimator of the mutual information between two ordinal vectors $x$ and $y$. Our approach is inductive (as opposed to deductive) in that it depends on the data generating distribution solely through some nonparametric…
We consider some formulations of the entropy bounds at the semiclassical level. The entropy S(V) localized in a region V is divergent in quantum field theory (QFT). Instead of it we focus on the mutual information I(V,W)=S(V)+S(W)-S(V\cup…
One of the crucial steps in scientific studies is to specify dependent relationships among factors in a system of interest. Given little knowledge of a system, can we characterize the underlying dependent relationships through observation…
Many interesting real-world systems are represented as complex networks with multiple types of interactions and complicated dependency structures between layers. These interactions can be encoded as having a valence with positive links…
In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known…
We introduce the Mutual Information Machine (MIM), a probabilistic auto-encoder for learning joint distributions over observations and latent variables. MIM reflects three design principles: 1) low divergence, to encourage the encoder and…
Mutual information (MI) is a general measure of statistical dependence with widespread application across the sciences. However, estimating MI between multi-dimensional variables is challenging because the number of samples necessary to…
Let $S_n^{(2)}$ denote the iterated partial sums. That is, $S_n^{(2)}=S_1+S_2+ ... +S_n$, where $S_i=X_1+X_2+ ... s+X_i$. Assuming $X_1, X_2,....,X_n$ are integrable, zero-mean, i.i.d. random variables, we show that the persistence…
A common concern when trying to draw causal inferences from observational data is that the measured covariates are insufficiently rich to account for all sources of confounding. In practice, many of the covariates may only be proxies of the…