Related papers: Entropy Inequalities for Sums in Prime Cyclic Grou…
We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…
We prove an inequality for the entropy numbers in terms of nonlinear Kolmogorov's widths. This inequality is in a spirit of known inequalities of this type and it is adjusted to the form convenient in applications for $m$-term…
We introduce a new, elementary method for studying random differences in arithmetic progressions and convergence phenomena along random sequences of integers. We apply our method to obtain significant improvements on previously known…
We consider the central limit theorem for stable laws in the case of the standardized sum of independent and identically distributed random variables with regular probability density function. By showing decay of different entropy…
In this paper, we establish some general forms of the law of the iterated logarithm for independent random variables in a sub-linear expectation space, where the random variables are not necessarily identically distributed. Exponential…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
We study the entropy production rate in systems described by linear Langevin equations, containing mixed even and odd variables under time reversal. Exact formulas are derived for several important quantities in terms only of the means and…
We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…
The R\'enyi and Shannon entropies are information-theoretic measures which have enabled to formulate the position-momentum uncertainty principle in a much more adequate and stringent way than the (variance-based) Heisenberg-like relation.…
We use the $f-divergence$ also called relative entropy as a measure of diversity between probability densities and review its basic properties. In the sequence we define a few objects which capture relevant information from the sample of a…
We study convexity properties of R\'{e}nyi entropy as function of $\alpha>0$ on finite alphabets. We also describe robustness of the R\'{e}nyi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on…
We investigate the role of convexity in R\'enyi entropy power inequalities. After proving that a general R\'enyi entropy power inequality in the style of Bobkov-Chistyakov (2015) fails when the R\'enyi parameter $r\in(0,1)$, we show that…
The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi…
We investigate the question of whether the entropy and the Renyi entropies of the vacuum state reduced to a region of the space can be represented in terms of correlators in quantum field theory. In this case, the positivity relations for…
For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…
The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings.…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
In this paper, we improve the known estimates for the invariance entropy of a nonlinear control system. For sets of complete approximate controllability we derive an upper bound in terms of Lyapunov exponents and for uniformly hyperbolic…
The notion of entropy appears in many fields and this paper is a survey about entropies in several branches of Mathematics. We are mainly concerned with the topological and the algebraic entropy in the context of continuous endomorphisms of…