相关论文: A new approach to mutual information

The definitions of delayed mutual information and multi-information are recalled. It is shown how the delayed mutual information may be used to reconstruct the interaction topology resulting from some unknown scale-free graph with its…

The authors transpose a discrete notion of indetermination coupling in the case of continuous probabilities. They show that this coupling, expressed on densities, cannot be captured by a specific copula which acts on cumulative distribution…

A new method to measure nonlinear dependence between two variables is described using mutual information to analyze the separate linear and nonlinear components of dependence. This technique, which gives an exact value for the proportion of…

Obtaining a closed-form sampling distribution for the coalescent with recombination is a challenging problem. In the case of two loci, a new framework based on asymptotic series has recently been developed to derive closed-form results when…

A convenient framework for dealing with asymptotic limit problems of probabilistic nature is provided. These problems include questions such as finding the asymptotic proportion of terms of a sequence falling inside a given interval, or the…

Given a set of independent Poisson random variables with common mean, we study the distribution of their maximum and obtain an accurate asymptotic formula to locate the most probable value of the maximum. We verify our analytic results with…

Estimating conditional mutual information (CMI) is an essential yet challenging step in many machine learning and data mining tasks. Estimating CMI from data that contains both discrete and continuous variables, or even discrete-continuous…

A collaborative distributed binary decision problem is considered. Two statisticians are required to declare the correct probability measure of two jointly distributed memoryless process, denoted by $X^n=(X_1,\dots,X_n)$ and…

Let $LA_{n}(\tau)$ be the length of the longest alternating subsequence of a uniform random permutation $\tau\in[n]$. Classical probabilistic arguments are used to rederive the asymptotic mean, variance and limiting law of $LA_{n}(\tau)$.…

Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion.…

Information estimates such as the ``direct method'' of Strong et al. (1998) sidestep the difficult problem of estimating the joint distribution of response and stimulus by instead estimating the difference between the marginal and…

Given an arbitrary continuous probability density function, it is introduced a conjugated probability density, which is defined through the Shannon information associated with its cumulative distribution function. These new densities are…

In this work, we attempt to refine the classic asymptotic formulae to describe the probability distribution of likelihood-ratio statistical tests. The idea is to split the probability distribution function into two parts. One part is…

The framework of Partial Information Decomposition (PID) unveils complex nonlinear interactions in network systems by dissecting the mutual information (MI) between a target variable and several source variables. While PID measures have…

In this note, we give an explicit expression for the quantile of a mixture of two random variables. We carefully examine all possible cases of discrete and continuous variables with possibly unbounded support. The result is useful for…

Describing statistical dependencies is foundational to empirical scientific research. For uncovering intricate and possibly non-linear dependencies between a single target variable and several source variables within a system, a principled…

We consider the classical problem of discrete distribution estimation using i.i.d. samples in a novel scenario where additional side information is available on the distribution. In large alphabet datasets such as text corpora, such side…

This paper is concerned with the asymptotic spreading of a Lotka-Volterra cooperative system. By using the theory of asymptotic spreading of nonautonomous equations, the asymptotic speeds of spreading of unknown functions formulated by a…

We consider a hidden Markov model, where the signal process, given by a diffusion, is only indirectly observed through some noisy measurements. The article develops a variational method for approximating the hidden states of the signal…

Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual…