Related papers: Mutual Information for the Stochastic Block Model …
Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number…
We develop numerical and analytical approaches to calculate mutual information between complete paths of two molecular components embedded into a larger reaction network. In particular, we focus on a continuous-time Markov chain formalism,…
In this letter we determine the derivative of the mutual information corresponding to bit-interleaved coded modulation systems. The derivative follows as a linear combination of minimum-mean-squared error functions of coded modulation sets.…
Vindicating a sophisticated but non-rigorous physics approach called the cavity method, we establish a formula for the mutual information in statistical inference problems induced by random graphs and we show that the mutual information…
Mutual information is widely used, in a descriptive way, to measure the stochastic dependence of categorical random variables. In order to address questions such as the reliability of the descriptive value, one must consider…
We study the generic scaling properties of the mutual information between two disjoint intervals, in a class of one-dimensional quantum critical systems described by the c=1 bosonic field theory. A numerical analysis of a spin-chain model…
Because of the kinematic reversibility of the Stokes equation, fluid mixing at the microscale requires an interplay between advection and diffusion. Here we introduce mutual information between particle positions before and after mixing as…
Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From…
Estimating mutual information between continuous random variables is often intractable and extremely challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual…
Consider random linear estimation with Gaussian measurement matrices and noise. One can compute infinitesimal variations of the mutual information under infinitesimal variations of the signal-to-noise ratio or of the measurement rate. We…
One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of…
Measuring model performance is a key issue for deep learning practitioners. However, we often lack the ability to explain why a specific architecture attains superior predictive accuracy for a given data set. Often, validation accuracy is…
The paper considers the problem of distributed adaptive linear parameter estimation in multi-agent inference networks. Local sensing model information is only partially available at the agents and inter-agent communication is assumed to be…
Estimating mutual information from observed samples is a basic primitive, useful in several machine learning tasks including correlation mining, information bottleneck clustering, learning a Chow-Liu tree, and conditional independence…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
A model to estimate the asymptotic isotropic mutual information of a multiantenna channel is considered. Using a block-based dynamics and the angle diversity of the system, we derived what may be thought of as the operator-valued version of…
Mutual Information (MI) is an useful tool for the recognition of mutual dependence berween data sets. Differen methods for the estimation of MI have been developed when both data sets are discrete or when both data sets are continuous. The…
This paper studies a high-dimensional inference problem involving the matrix tensor product of random matrices. This problem generalizes a number of contemporary data science problems including the spiked matrix models used in sparse…
We consider the problem of recovering the community structure in the stochastic block model. We aim to describe the mutual information between the observed network and the actual community structure as the number of nodes diverges while the…
Factorizing low-rank matrices has many applications in machine learning and statistics. For probabilistic models in the Bayes optimal setting, a general expression for the mutual information has been proposed using heuristic statistical…