Related papers: Estimating differential entropy using recursive co…
We study an unbiased estimator for the density of a sum of random variables that are simulated from a computer model. A numerical study on examples with copula dependence is conducted where the proposed estimator performs favourably in…
Variable selection is an important problem in statistics and machine learning. Copula Entropy (CE) is a mathematical concept for measuring statistical independence and has been applied to variable selection recently. In this paper we…
Statistical independence and conditional independence are two fundamental concepts in statistics and machine learning. Copula Entropy is a mathematical concept defined by Ma and Sun for multivariate statistical independence measuring and…
The relative entropy for two different degenerate diffusion processes is estimated by using the Wasserstein distance of initial distributions and the difference between coefficients. As applications, the entropy cost inequality and…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
A novel method for correlation analysis using scale-dependent Renyi entropies is described. The method involves calculating the entropy of a data distribution as an explicit function of the scale of a d-dimensional partition of d-cubes,…
We propose a novel method for testing isotropy of a three-dimensional distribution using Shannon entropy. We test the method on some Monte Carlo simulations of isotropic and anisotropic distributions and find that the method can effectively…
Gaussian copulas are widely used to estimate multivariate distributions and relationships. We present algorithms for estimating Gaussian copula correlations that ensure differential privacy. We first convert data values into sets of two-way…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…
This paper addresses the challenge of identifying a minimal subset of discrete, independent variables that best predicts a binary class. We propose an efficient iterative method that sequentially selects variables based on which one…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
A dependence measure for arbitrary type pairs of random variables is proposed and analyzed, which in the particular case where both random variables are continuous turns out to be a concordance measure. Also, a sample version of the…
We investigate the validity of two resampling techniques when carrying out inference on the underlying unknown copula using a recently proposed class of smooth, possibly data-adaptive nonparametric estimators that contains empirical…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
This paper introduces an innovative method for constructing copula models capable of describing arbitrary non-monotone dependence structures. The proposed method enables the creation of such copulas in parametric form, thus allowing the…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…
By applying Sklar's theorem to the Multivariate Bernoulli Distribution (MBD), this paper proposes a framework to decouple marginal distributions from the dependence structure, clarifying interactions among binary variables. Explicit…
Our article addresses the problem of flexibly estimating a multivariate density while also attempting to estimate its marginals correctly. We do so by proposing two new estimators that try to capture the best features of mixture of normals…