Related papers: Two Measures of Dependence
While learning with limited labelled data can improve performance when the labels are lacking, it is also sensitive to the effects of uncontrolled randomness introduced by so-called randomness factors (e.g., varying order of data). We…
We study distributional similarity measures for the purpose of improving probability estimation for unseen cooccurrences. Our contributions are three-fold: an empirical comparison of a broad range of measures; a classification of similarity…
We characterize the growth of the Sibson and Arimoto mutual informations and $\alpha$-maximal leakage, of any order that is at least unity, between a random variable and a growing set of noisy, conditionally independent and…
Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…
Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
It is often necessary to make sampling-based statistical inference about many probability distributions in parallel. Given a finite computational resource, this article addresses how to optimally divide sampling effort between the samplers…
We consider two types of entropy, namely, Shannon and R\'{e}nyi entropies of the Poisson distribution, and establish their properties as the functions of intensity parameter. More precisely, we prove that both entropies increase with…
Measuring conditional dependencies among the variables of a network is of great interest to many disciplines. This paper studies some shortcomings of the existing dependency measures in detecting direct causal influences or their lack of…
Entanglement R\'enyi-$\alpha$ entropy is an entanglement measure. It generalizes the entanglement of formation, and they coincide when $\alpha$ tends to 1. We derive analytical lower and upper bounds for the entanglement R\'enyi-$\alpha$…
The diversity of the symbols of the information source is calculated following the definition that entropy is the information loss and following a new entropy-symbol similarity relation after the rejection of the Gibbs paradox statement.…
In multivariate analysis, uncertainty arises from two sources: the marginal distributions of the variables and their dependence structure. Quantifying the dependence structure is crucial, as it provides valuable insights into the…
We introduce novel information-entropic variables -- a Point Divergence Gain (${\Omega}^{(l \rightarrow m)}_\alpha$), a Point Divergence Gain Entropy ($I_\alpha$), and a Point Divergence Gain Entropy Density ($P_\alpha$) -- which are…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
We provide a condition under which a version of Shannon's Entropy Power Inequality will hold for dependent variables. We provide information inequalities extending those found in the independent case.
As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
We report an inconsistency found in probability theory (also referred to as measure-theoretic probability). For probability measures induced by real-valued random variables, we deduce an "equality" such that one side of the "equality" is a…
Shannon and Renyi entropies are quantitative measures of uncertainty in a data set. They are developed by Renyi in the context of entropy theory. These measures have been studied in the case of the multivariate t-distributions. We extend…
When multiple investigators analyze a common dataset, the data reuse induces dependence across testing procedures, affecting the distribution of errors. Existing techniques of managing dependent tests require either cross-study coordination…