Related papers: Bounds for some entropies and special functions
We consider the generalized differential entropy of normalized sums of independent and identically distributed (IID) continuous random variables. We prove that the R\'{e}nyi entropy and Tsallis entropy of order $\alpha\ (\alpha>0)$ of the…
Examples of joint probability distributions are studied in terms of Tsallis' nonextensive statistics both for correlated and uncorrelated variables, in particular it is explicitely shown how correlations in the system can make Tsallis…
Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann- Gibbs form of the entropy ensures that…
The purpose of this note is to give the general solution of two functional equations connected to the Shannon entropy and also to the Tsallis entropy. As a result of this, we present the regular solution of these equations, as well.…
We consider the two-dimensional (2d) Ising model on a infinitely long cylinder and study the probabilities $p_i$ to observe a given spin configuration $i$ along a circular section of the cylinder. These probabilities also occur as…
We examine a generalization of the binomial distribution associated with a strictly increasing sequence of numbers and we prove its Poisson-like limit. Such generalizations might be found in quantum optics with imperfect detection. We…
In many applications, the probability density function is subject to experimental errors. In this work the continuos dependence of a class of generalized entropies on the experimental errors is studied. This class includes the C. Shannon,…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis…
A family of nonlinear ordinary differential equations with arbitrary order is obtained by using nonextensive concepts related to the Tsallis entropy. Applications of these equations are given here. In particular, a connection between…
We generalise the known fact that for binomial $X_{n,k} \sim \mathrm{Bin}(n, k/n)$ one has $\inf_{k>1,n} \mathrm{P}(X_{n,k} \geq k) \geq \lim_{k \to 1+}\mathrm{P}(X_{2,k} \geq k) = 1/4$ to cover probabilities of exceeding a constant shift…
Estimates are constructed for the deviation of the concentration functions of sums of independent random variables with finite variances from the folded normal distribution function without any assumptions concerning the existence of the…
Motivated, in part, by the desire to develop an information-theoretic foundation for compound Poisson approximation limit theorems (analogous to the corresponding developments for the central limit theorem and for simple Poisson…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
In a previous paper: A. Paszkiewicz, T. Sobieszek, Additive Entropies of Partitions, we have given a description of additive partition entropies that is real functions $I$ on the set of finite partitions that are additive on stochastically…
In this paper, we study entropic uncertainty relations on a finite-dimensional Hilbert space and provide several tighter bounds for multi-measurements, with some of them also valid for R\'{e}nyi and Tsallis entropies besides the Shannon…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…