Related papers: Shannon Entropy and Kullback-Leibler Divergence in…
We introduce two classes of multivariate log skewed distributions with normal kernel: the log canonical fundamental skew-normal (log-CFUSN) and the log unified skew-normal (log-SUN). We also discuss some properties of the log-CFUSN family…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
For generic systems exhibiting power law behaviors, and hence multiscale dependencies, we propose a new, and yet simple, tool to analyze multifractality and intermittency, after noticing that these concepts are directly related to the…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
We quantify the degree of inhomogeneity in the Luminous Red Galaxy (LRG) distribution from the SDSS DR7 as a function of length scales by measuring the Shannon entropy in independent and regular cubic voxels of increasing grid sizes. We…
The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the…
Cubic transmuted (CT) distributions were introduced recently by \cite{granzotto2017cubic}. In this article, we derive Shannon entropy, Gini's mean difference and Fisher information (matrix) for CT distributions and establish some of their…
Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with…
The classical Maximum-Entropy Principle (MEP) based on Shannon entropy is widely used to construct least-biased probability distributions from partial information. However, the Shore-Johnson axioms that single out the Shannon functional…
In environmental studies, many data are typically skewed and it is desired to have a flexible statistical model for this kind of data. In this paper, we study a class of skewed distributions by invoking arguments as described by Ferreira…
In this paper we propose a family of multivariate asymmetric distributions over an arbitrary subset of set of real numbers which is defined in terms of the well-known elliptically symmetric distributions. We explore essential properties,…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…
We give sufficient identifiability conditions for estimating mixing proportions in two-component mixtures of skew normal distributions with one known component. We consider the univariate case as well as two multivariate extensions: a…
We give a detailed analysis of the Gibbs-type entropy notion and its dynamical behavior in case of time-dependent continuous probability distributions of varied origins: related to classical and quantum systems. The purpose-dependent usage…
Mixtures of factor analyzers (MFA) provide a powerful tool for modelling high-dimensional datasets. In recent years, several generalizations of MFA have been developed where the normality assumption of the factors and/or of the errors was…
This paper describes the Difference-of-Log-Normals (DLN) distribution. A companion paper makes the case that the DLN is a fundamental distribution in nature, and shows how a simple application of the CLT gives rise to the DLN in many…
The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence…