相关论文: Mutual information matrices are not always positiv…

Motivated by the need, in some Bayesian likelihood free inference problems, of imputing a multivariate counting distribution based on its vector of means and variance-covariance matrix, we define a generic multivariate discrete…

This thesis uses a quantity that is defined and justified by information theory -- mutual information -- to examine models of condensed matter systems. More precisely, it studies models which are made up out of ferromagnetically interacting…

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections, a problem relevant in compressed sensing, sparse superposition codes or code division multiple access just to cite few. There has…

We give a characterization for the extreme points of the convex set of correlation matrices with a countable index set. A Hermitian matrix is called a correlation matrix if it is positive semidefinite with unit diagonal entries. Using the…

We consider the problem of quantifying the information shared by a pair of random variables $X_{1},X_{2}$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left…

Let $M_{n}$ denote a random symmetric $n\times n$ matrix, whose entries on and above the diagonal are i.i.d. Rademacher random variables (taking values $\pm 1$ with probability $1/2$ each). Resolving a conjecture of Vu, we prove that the…

The ratio $P(S_n=x)/P(Z_n=x)$ is investigated for three cases: (a) when $S_n$ is a sum of 1-dependent non-negative integer-valued random variables (rvs), satisfying some moment conditions, and $Z_n$ is Poisson rv; (b) when $S_n$ is a…

We observe an infinite sequence of independent identically distributed random variables $X_1,X_2,\ldots$ drawn from an unknown distribution $p$ over $[n]$, and our goal is to estimate the entropy $H(p)=-\mathbb{E}[\log p(X)]$ within an…

We study the distribution of entries of a random permutation matrix under a "randomized basis," i.e., we conjugate the random permutation matrix by an independent random orthogonal matrix drawn from Haar measure. It is shown that under…

We consider the notion of the matrix (tensor) distribution of a measurable function of several variables. On the one hand, it is an invariant of this function with respect to a certain group of transformations of variables; on the other…

Let $p$ be a polynomial in the non-commuting variables $(a,x)=(a_1,...,a_{g_a},x_1,...,x_{g_x})$. If $p$ is convex in the variables $x$, then $p$ has degree two in $x$ and moreover, $p$ has the form $p = L + \Lambda ^T \Lambda,$ where $L$…

Let $Z$ be a random variable with values in a proper closed convex cone $C\subset \mathbb{R}^d$, $A$ a random endomorphism of $C$ and $N$ a random integer. We assume that $Z$, $A$, $N$ are independent. Given $N$ independent copies…

Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…

In this article we derive the best possible upper bound for $E[\max{X_i}-\min_i{X_i}]$ under given means and variances on $n$ random variables $X_i$. The random vector $(X_1,...,X_n)$ is allowed to have any dependence structure, provided $E…

Mutual information is widely applied to learn latent representations of observations, whilst its implication in classification neural networks remain to be better explained. We show that optimising the parameters of classification neural…

Mutual information and information entropies in momentum space are proposed as measures of the non-local aspects of information. Singlet and triplet state members of the helium isoelectronic series are employed to examine Coulomb and Fermi…

Exponential models of distributions are widely used in machine learning for classiffication and modelling. It is well known that they can be interpreted as maximum entropy models under empirical expectation constraints. In this work, we…

Mutual Information (MI) and Conditional Mutual Information (CMI) are multi-purpose tools from information theory that are able to naturally measure the statistical dependencies between random variables, thus they are usually of central…

Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…

We show the following version of the Schur's product theorem. If $M=(M_{j,k})_{j,k=1}^n\in{\mathbb R}^{n\times n}$ is a positive semidefinite matrix with all entries on the diagonal equal to one, then the matrix $N=(N_{j,k})_{j,k=1}^n$ with…