Related papers: A Perfect Set of Reals with Finite Self-Informatio…
Mutual information I in infinite sequences (and in their finite prefixes) is essential in theoretical analysis of many situations. Yet its right definition has been elusive for a long time. I address it by generalizing Kolmogorov Complexity…
Symmetry of information establishes a relation between the information that x has about y (denoted I(x : y)) and the information that y has about x (denoted I(y : x)). In classical information theory, the two are exactly equal, but in…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
The information in an individual finite object (like a binary string) is commonly measured by its Kolmogorov complexity. One can divide that information into two parts: the information accounting for the useful regularity present in the…
We calculate the mutual information (MI) of a two-layered neural network with noiseless, continuous inputs and binary, stochastic outputs under several assumptions on the synaptic efficiencies. The interesting regime corresponds to the…
Consider a grade 2 perfect ideal $I$ in $R=k[x_1,\cdots,x_d]$ which is generated by forms of the same degree. Assume that the presentation matrix $\varphi$ is almost linear, that is, all but the last column of $\varphi$ consist of entries…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
A set A of positive integers is called a perfect difference set if every nonzero integer has an unique representation as the difference of two elements of A. We construct dense perfect difference sets from dense Sidon sets. As a consequence…
We revisit the question of modeling incomplete information among 2 Bayesian players, following an ex-ante approach based on values of zero-sum games. $K$ being the finite set of possible parameters, an information structure is defined as a…
Let $\mathcal{V}$ be the set of all combinations of expected value of finite objective functions from designing information. I showed that $\mathcal{V}$ is a compact and convex set implemented by signal structures with finite support when…
Let $\beta>1$. We define a class of similitudes \[S:=\left\{f_{i}(x)=\dfrac{x}{\beta^{n_i}}+a_i:n_i\in \mathbb{N}^{+}, a_i\in \mathbb{R}\right\}.\] Taking any finite similitudes $\{f_{i}(x)\}_{i=1}^{m} $ from $S$, it is well known that…
This paper introduces the notion of exact common information, which is the minimum description length of the common randomness needed for the exact distributed generation of two correlated random variables $(X,Y)$. We introduce the quantity…
Lieb and Ruskai's strong subadditivity theorem, which shows that the conditional mutual information must be nonnegative, is fundamental in quantum theory. It has numerous applications, such as in quantum error correction. When the mutual…
Information (I) is defined as the amount of the data after data compression. The first law of information theory: the total amount of data L (the sum of entropy S and information I) of an isolated system remains unchanged. The second law of…
We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings $x$ and $y$ is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having $x$ and the…
Multivariate mutual information provides a conceptual framework for characterizing higher-order interactions in complex systems. Two well-known measures of multivariate information---total correlation and dual total correlation---admit a…
A square-free monomial ideal $I$ is called an {\it $f$-ideal}, if both $\delta_{\mathcal{F}}(I)$ and $\delta_{\mathcal{N}}(I)$ have the same $f$-vector, where $\delta_{\mathcal{F}}(I)$ ($\delta_{\mathcal{N}}(I)$, respectively) is the facet…
Let $\mathbb{K}$ be a finite commutative ring, and let $\mathbb{L}$ be a commutative $\mathbb{K}$-algebra. Let $A$ and $B$ be two $n \times n$-matrices over $\mathbb{L}$ that have the same characteristic polynomial. The main result of this…
For discrete random variables X_1,..., X_n we construct an n by n matrix. In the (i,j) entry we put the mutual information I(X_i;X_j) between X_i and X_j. In particular, in the (i,i) entry we put the entropy H(X_i)=I(X_i;X_i) of X_i. This…
We consider the (Renyi) mutual information, $I^{(n)}(A,B) = S^{(n)}_A+S^{(n)}_{B} - S^{(n)}_{A \cup B}$, of distant compact spatial regions A and B in the vacuum state of a free scalar field. The distance r between A and B is much greater…