Related papers: Dependence and Uncertainty: Information Measures u…
Asymmetry is an inherent property of bivariate associations and therefore must not be ignored. The currently applicable dependence measures mask the potential asymmetry of the underlying dependence structure by implicitly assuming that…
We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution…
As a fundamental concept in information theory, mutual information ($MI$) has been commonly applied to quantify association between random vectors. Most existing nonparametric estimators of $MI$ have unstable statistical performance since…
Uncertainty reduction is vital for improving system reliability and reducing risks. To identify the best target for uncertainty reduction, uncertainty importance measure is commonly used to prioritize the significance of input variable…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…
The information shared among observables representing processes of interest is traditionally evaluated in terms of macroscale measures characterizing aggregate properties of the underlying processes and their interactions. Traditional…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
Following our previous work on copula-based nonsymmetric dependence measures, we introduce similar measures for discrete random variables. The measures cover the range between two extremes: independence and complete dependence, which take…
Tsallis' non-extensive entropy is extended to incorporate the dependence on affinities between the microstates of a system. At the core of our construction of the extended entropy ($\mathcal{H}$) is the concept of the effective number of…
Estimation of mutual information between random variables has become crucial in a range of fields, from physics to neuroscience to finance. Estimating information accurately over a wide range of conditions relies on the development of…
One of the crucial steps in scientific studies is to specify dependent relationships among factors in a system of interest. Given little knowledge of a system, can we characterize the underlying dependent relationships through observation…
The dynamics of a power system with a significant presence of renewable energy resources are growing increasingly nonlinear. This nonlinearity is a result of the intermittent nature of these resources and the switching behavior of their…
Separability conditions for a bipartite quantum system of finite-dimensional subsystems are formulated in terms of R\'{e}nyi and Tsallis entropies. Entropic uncertainty relations often lead to entanglement criteria. We propose new approach…
Medical image registration plays an important role in determining topographic and morphological changes for functional diagnostic and therapeutic purposes. Manual alignment and semi-automated software still have been used; however they are…
Algorithmic entropy and Shannon entropy are two conceptually different information measures, as the former is based on size of programs and the later in probability distributions. However, it is known that, for any recursive probability…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
We introduce a new test procedure of independence in the framework of parametric copulas with unknown marginals. The method is based essentially on the dual representation of $\chi^2$-divergence on signed finite measures. The asymptotic…
Understanding cause-effect relationships is a crucial part of the scientific process. As Bell's theorem shows, within a given causal structure, classical and quantum physics impose different constraints on the correlations that are…
We show that different entropic measures of fluctuations lead to contradictory uncertainty relations for two complementary observables. We apply Tsallis and R\'{e}nyi entropies to the joint distribution emerging from a noisy simultaneous…