Related papers: Ergodic MIMO Mutual Information: Twenty Years Afte…
In this review, we investigate a key information-theoretic performance metric in multiple-antenna wireless communications, the so-called outage probability. This quantity may be expressed in terms of a moment generating function, for which…
In the setting where information cannot be verified, we propose a simple yet powerful information theoretical framework---the Mutual Information Paradigm---for information elicitation mechanisms. Our framework pays every agent a measure of…
In this paper we derive the exact analytical expressions for the information and covariance matrices of the multivariate Burr and related distributions. These distributions arise as tractable parametric models in reliability, actuarial…
Estimating mutual correlations between random variables or data streams is essential for intelligent behavior and decision-making. As a fundamental quantity for measuring statistical relationships, mutual information has been extensively…
The generalized logarithmic electrodynamics with two parameters $\beta$ and $\gamma$ is considered. The indexes of refraction of light in the external magnetic field are calculated. In the case $\beta=\gamma$ we come to results obtained by…
We formulate and analyze the compound information bottleneck programming. In this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow \mathsf{Z} $ is assumed with fixed marginal distributions $\mathsf{P}_{\mathsf{X}}$…
We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI…
In order to meet the demands of future innovative applications, many efforts have been made to exceed the limits predicted by Shannon's Theory. Besides the investigation of beyond-Shannon metrics such as security, latency, and semantics,…
Multivariate mutual information provides a conceptual framework for characterizing higher-order interactions in complex systems. Two well-known measures of multivariate information---total correlation and dual total correlation---admit a…
Iterated commutators of multilinear Calderon-Zygmund operators and pointwise multiplication with functions in $BMO$ are studied in products of Lebesgue spaces. Both strong type and weak end-point estimates are obtained, including weighted…
In this paper, the third in a series, we continue to study the generalized Lam\'{e} equation H$(n_0,n_1,n_2,n_3;B)$ with the Darboux-Treibich-Verdier potential \begin{equation*} y^{\prime \prime }(z)=\bigg[…
In this contribution, we propose a generic online (also sometimes called adaptive or recursive) version of the Expectation-Maximisation (EM) algorithm applicable to latent variable models of independent observations. Compared to the…
In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the…
With contemporary data sets becoming too large to analyze the data directly, various forms of aggregated data are becoming common. The original individual data are points, but after aggregation, the observations are interval-valued (e.g.).…
A new trend in deep learning, represented by Mutual Information Neural Estimation (MINE) and Information Noise Contrast Estimation (InfoNCE), is emerging. In this trend, similarity functions and Estimated Mutual Information (EMI) are used…
Historically, the polylogarithm has attracted specialists and non-specialists alike with its lovely evaluations. Much the same can be said for Euler sums (or multiple harmonic sums), which, within the past decade, have arisen in…
This is a summary for the authors' article "The formal KZ equation on the moduli space ${\mathcal M}_{0,5}$ and the harmonic product of multiple zeta values" (prerint (2009) arXiv:0910.0718), including a new result on the five term relation…
The I-MMSE formula connects two important quantities in information theory and estimation theory: the mutual information and the minimum mean-squared error (MMSE). It states that in a scalar Gaussian channel, the derivative of the mutual…
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data. This comes with several immediate…
Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data. This class of estimators is referred to as neural…