相关论文: Existence and Continuity of Differential Entropy f…

We prove an entropy version of van der Corput's difference theorem: the entropy of a sequence is equal to the entropy of its differences. This reveals a potential correspondence between the theory of uniform distribution mod 1 and entropy.…

We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with…

The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…

The paper search for the minimum of the entropy of a two- dimensional distribution in the Fr\'echet class, the class of distributions with given marginals. The main result for discrete distributions is an algorithm for building the…

We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…

We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…

In multiple classification, one aims to determine whether a testing sequence is generated from the same distribution as one of the M training sequences or not. Unlike most of existing studies that focus on discrete-valued sequences with…

We study probability distributions of convergent random series of a special structure, called perpetuities. By giving a new argument, we prove that such distributions are of pure type: degenerate, absolutely continuous, or continuously…

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

We consider some classes of piecewise expanding maps in finite dimensional spaces having invariant probability measures which are absolutely continuous with respect to Lebesgue measure. We derive an entropy formula for such measures and,…

Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…

We introduce a family of scale-invariant entropy statistics derived from logarithmically aggregated distance distributions of point processes, with prime numbers serving as a motivating example. The construction associates to each finite…

We study a new class of so-called rational-infinitely (or quasi-infinitely) divisible probability laws on the real line. The characteristic functions of these distributions are ratios of the characteristic functions of classical infinitely…

This paper serves a twofold purpose. First, a unified perspective on diversity indices is introduced based on an entropic basis. It is shown that the class of all linear combinations of the entropic basis, referred to as the class of linear…

We apply the Law of Total Probability to the construction of scale-invariant probability distribution functions (pdfs), and require that probability measures be dimensionless and unitless under a continuous change of scales. If the…

Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…

We consider the differential entropy of probability measures absolutely continuous with respect to a given $\sigma$-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general…

Stimulated by the need of describing useful notions related to information measures, we introduce the `pdf-related distributions'. These are defined in terms of transformation of absolutely continuous random variables through their own…

A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…

We introduce the notion of uniform exactness, or uniform amenability at infinity, for discrete groups and prove it for a wide class of groups containing free groups and their limit groups. This shows a novel strong convergence phenomenon…