Related papers: Evaluation of mutual information estimators on non…
The mutual information characterizes correlations between spatially separated regions of a system. Yet, in experiments we often measure dynamical correlations, which involve probing operators that are also separated in time. Here, we…
Mutual Information (MI) is a powerful statistical measure that quantifies shared information between random variables, particularly valuable in high-dimensional data analysis across fields like genomics, natural language processing, and…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
In this manuscript we present a comparative study about the determination of the relaxation (\textit{i.e.}, independence) time scales obtained from the correlation function, the mutual information, and a criterion based on the evaluation of…
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precisely the mutual…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
Non-equilibrium systems have long-ranged spatial correlations even far away from critical points. This implies that the likelihoods of spatial steady state profiles of physical observables are nonlocal functionals. In this letter, it is…
We define two minimum distance estimators for dependent data by minimizing some approximated Maximum Mean Discrepancy distances between the true empirical distribution of observations and their assumed (parametric) model distribution. When…
We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of…
The Mutual Information (MI) is an often used measure of dependency between two random variables utilized in information theory, statistics and machine learning. Recently several MI estimators have been proposed that can achieve parametric…
Renormalization is an essential technique in field-theoretic descriptions of natural phenomena, where the absence of a UV-complete description yields an abundance of divergent quantities. While the renormalization prescription has been…
Recently, global pulsar timing arrays have released results from searching for a nano-Hertz gravitational wave background signal. Although there has not been any definite evidence of the presence of such a signal in residuals of pulsar…
In this paper, we study measures of quantum non-Markovianity based on the conditional mutual information. We obtain such measures by considering multiple parts of the total environment such that the conditional mutual information can be…
While the linear Pearson correlation coefficient represents a well-established normalized measure to quantify the interrelation of two stochastic variables $X$ and $Y$, it fails for multidimensional variables such as Cartesian coordinates.…
Mutual information (MI) is a general measure of statistical dependence with widespread application across the sciences. However, estimating MI between multi-dimensional variables is challenging because the number of samples necessary to…
This paper is concerned with the problem of distributed estimation for time-varying interconnected dynamic systems with arbitrary coupling structures. To guarantee the robustness of the designed estimators, novel distributed stability…
Marginal structural models are a popular method for estimating causal effects in the presence of time-varying exposures. In spite of their popularity, no scalable non-parametric estimator exist for marginal structural models with…
Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of…
We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI…
We construct and analyze an estimator of association between random variables based on their similarity in both direction and magnitude. Under special conditions, the proposed measure becomes a robust and consistent estimator of the linear…