相关论文: Mutual information is copula entropy

Reshef et al. recently proposed a new statistical measure, the "maximal information coefficient" (MIC), for quantifying arbitrary dependencies between pairs of stochastic quantities. MIC is based on mutual information, a fundamental…

We study the convexity of mutual information along the evolution of the heat equation. We prove that if the initial distribution is log-concave, then mutual information is always a convex function of time. We also prove that if the initial…

Estimating the mutual information from samples from a joint distribution is a challenging problem in both science and engineering. In this work, we realize a variational bound that generalizes both discriminative and generative approaches.…

Relational data augmentation is a powerful technique for enhancing data analytics and improving machine learning models by incorporating columns from external datasets. However, it is challenging to efficiently discover relevant external…

We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution. This quantity is based on a recently proposed…

Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current…

We study conditional mutual information (cMI) between a pair of variables $X,Y$ given a third one $Z$ and derived quantities including transfer entropy (TE) and causation entropy (CE) in the dynamically relevant context where $X=T(Y,Z)$ is…

By combining a bound on the absolute value of the difference of mutual information between two joint probablity distributions with a fixed variational distance, and a bound on the probability of a maximal deviation in variational distance…

We develop a supervised deep-learning approach to estimate mutual information between two continuous random variables. As labels, we use the Linfoot informational correlation, a transformation of mutual information that has many important…

The conditional mutual information quantifies the conditional dependence of two random variables. It has numerous applications; it forms, for example, part of the definition of transfer entropy, a common measure of the causal relationship…

We investigate the dynamics of information among the parties of tripartite systems. We start by proving two results concerning the monogamy of mutual information. The first one states that mutual information is monogamous for generic…

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…

The integrated information theory is thought to be a key clue towards the theoretical understanding of consciousness. In this study, we propose a simple numerical model comprising a set of coupled double quantum dots, where the…

In this paper we have studied the effect of deformation and temperature on holographic entanglement entropy and mutual information between two subsystems in a deformed field theory at finite temperature. The $T{\overline{T}}$ deformation…

Estimating mutual information accurately is pivotal across diverse applications, from machine learning to communications and biology, enabling us to gain insights into the inner mechanisms of complex systems. Yet, dealing with…

The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…

In this manuscript we present a comparative study about the determination of the relaxation (\textit{i.e.}, independence) time scales obtained from the correlation function, the mutual information, and a criterion based on the evaluation of…

We compute the entanglement entropy and mutual information for two disjoint intervals in two-dimensional conformal field theories as a function of time after a local quench, using the replica trick and boundary conformal field theory. We…

The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…

In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…