Related papers: On the Monotonicity of the Copula Entropy
We explore a few common models on how correlations affect information. The main model considered is the Shannon mutual information $I(S:R_1,\cdots, R_i)$ over distributions with marginals $P_{S,R_i}$ fixed for each $i$, with the analogy in…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…
This paper presents an introduction to the stochastic concepts of \emph{coupling} and \emph{copula}. Coupling means the construction of a joint distribution of two or more random variables that need not be defined on one and the same…
The information shared among observables representing processes of interest is traditionally evaluated in terms of macroscale measures characterizing aggregate properties of the underlying processes and their interactions. Traditional…
Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…
Quantitative studies in many fields involve the analysis of multivariate data of diverse types, including measurements that we may consider binary, ordinal and continuous. One approach to the analysis of such mixed data is to use a copula…
Quantification of microbial interactions from 16S rRNA and meta-genomic sequencing data is difficult due to their sparse nature, as well as the fact that the data only provides measures of relative abundance. In this paper, we propose using…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
Verification and validation of fully automated vehicles is linked to an almost intractable challenge of reflecting the real world with all its interactions in a virtual environment. Influential stochastic parameters need to be extracted…
How does the information flow between different brain regions during various stimuli? This is the question we aim to address by studying complex cognitive paradigms in terms of Information Theory. To assess creativity and the emergence of…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
The purpose of this paper is to characterize the concept of monotonicity according to a direction related to a set of n random variables in terms of its associated n-copula C. We start establishing relationships in the bivariate and…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…
Entropic measures provide analytic tools to help us understand correlation in quantum systems. In our previous work, we calculated linear entropy and von Neumann entropy as entanglement measures for the ground state and lower lying excited…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
Copulas are mathematical objects that fully capture the dependence structure among random variables and hence, offer a great flexibility in building multivariate stochastic models. In statistics, a copula is used as a general way of…
Dependence strucuture estimation is one of the important problems in machine learning domain and has many applications in different scientific areas. In this paper, a theoretical framework for such estimation based on copula and copula…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…