相关论文: Entropy of the Sum of Two Independent, Non-Identic…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

The distribution of the sum of independent identically distributed uniform random variables is well-known. However, it is sometimes necessary to analyze data which have been drawn from different uniform distributions. By inverting the…

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

Exact formulas are derived for the probability density functions of the sum and difference of two independent non-central gamma distributed random variables, with both series and integral representations of the density presented. These…

We consider the distribution of the sum and the maximum of a collection of independent exponentially distributed random variables. The focus is laid on the explicit form of the density functions (pdf) of non-i.i.d. sequences. Those are…

A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…

We present an entropy comparison result concerning weighted sums of independent and identically distributed random variables.

We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…

Let $\mathsf{N}_{\rm d}\left[X\right]=\frac{1}{2\pi {\rm e}}{\rm e}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this…

This paper re-examines the density for sums of independent exponential, Erlang and gamma random variables. By using a divided difference perspective, the paper provides a unified approach to finding closed-form formulae for such…

We present an analytic method for computing the moments of a sum of independent and identically distributed random variables. The limiting behavior of these sums is very important to statistical theory, and the moment expressions that we…

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…

The (general) hypoexponential distribution is the distribution of a sum of independent exponential random variables. We consider the particular case when the involved exponential variables have distinct rate parameters. We prove that the…

The form invariance of pseudoadditivity is shown to determine the structure of nonextensive entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive information content, similar to the definition of…

We give a simple inequality for the sum of independent bounded random variables. This inequality improves on the celebrated result of Hoeffding in a special case. It is optimal in the limit where the sum tends to a Poisson random variable.

The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have…

We construct a non - improved exponential bounds for distribution of normed sums of i.,i.d. random variables with random numbers of summand.

This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…

We derive an asymptotic expansion for the distribution of a compound sum of independent random variables, all having the same light-tailed subexponential distribution. The examples of a Poisson and geometric number of summands serve as an…

We propose an entropy function for simplicial complices. Its value gives the expected cost of the optimal encoding of sequences of vertices of the complex, when any two vertices belonging to the same simplex are indistinguishable. We show…