相关论文: Submodularity of Mutual Information for Multivaria…
Recent studies have explored theoretically the ability of populations of neurons to carry information about a set of stimuli, both in the case of purely discrete or purely continuous stimuli, and in the case of multidimensional continuous…
Inspired by works on information transmission through quantum channels, we propose the use of a couple of mutual entropies to quantify the efficiency of continual measurement schemes in extracting information on the measured quantum system.…
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough…
We investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which…
We introduce a new concept called as the mutual uncertainty between two observables in a given quantum state which enjoys similar features like the mutual information for two random variables. Further, we define the conditional uncertainty…
Taking a functional approach, we derive a general expression for the gradient of the Mutual Information (MI) with respect to the system parameters in the stochastic systems. This expression covers the cases in which the system input depends…
Multiplicative noise arises in inverse problems when, for example, uncertainty on measurements is proportional to the size of the measurement itself. The likelihood that arises is hence more complicated than that from additive noise. We…
Estimating mutual information (MI) between two continuous random variables $X$ and $Y$ allows to capture non-linear dependencies between them, non-parametrically. As such, MI estimation lies at the core of many data science applications.…
We extend previous results on covert communication over the additive white Gaussian noise channel to two other types of additive noise channels. The first is the Gaussian channel with memory, where the noise sequence is a Gaussian vector…
Existing studies on self-supervised speech representation learning have focused on developing new training methods and applying pre-trained models for different applications. However, the quality of these models is often measured by the…
In this work we present a new method for the estimation of Mutual Information (MI) between random variables. Our approach is based on an original interpretation of the Girsanov theorem, which allows us to use score-based diffusion models to…
Mutual information is the reciprocal information that is common to or shared by two or more parties. Quantum mutual information for bipartite quantum systems is non-negative, and bears the interpretation of total correlation between the two…
In our companion paper [1], an information identity decomposition has been derived, which can be interpreted as a law of conservation of information flows in feedback systems. In this paper, we further investigate this decomposition result…
We studied the mutual information between a stimulus and a large system consisting of stochastic, statistically independent elements that respond to a stimulus. The Mutual Information (MI) of the system saturates exponentially with system…
Studying the typical entanglement entropy of a bipartite system when averaging over different ensembles of pure quantum states has been instrumental in different areas of physics, ranging from many-body quantum chaos to black hole…
The intensity statistics of signals in the presence of Gaussian noise is obtained by studying the model of a random signal plus a random phasor sum. The additive Gaussian noise is shown to result in a Bessel transform of the probability…
In recent years, there has been a significant increase in applications of multimodal signal processing and analysis, largely driven by the increased availability of multimodal datasets and the rapid progress in multimodal learning systems.…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
In this work we study the opinion evolution in a community-based population with intergroup interactions. We address two issues. First, we consider that such intergroup interactions can be negative with some probability $p$. We develop a…
Estimating Mutual Information (MI), a key measure of dependence of random quantities without specific modelling assumptions, is a challenging problem in high dimensions. We propose a novel mutual information estimator based on parametrizing…