Related papers: An Effective Bernstein-type Bound on Shannon Entro…
It is proven that a conjecture of Tao (2010) holds true for log-concave random variables on the integers: For every $n \geq 1$, if $X_1,\ldots,X_n$ are i.i.d. integer-valued, log-concave random variables, then $$ H(X_1+\cdots+X_{n+1}) \geq…
A probabilistic source is defined as the set of infinite words (over a given denumerable alphabet) endowed with a probability $\mu$. The paper deals with general binary sources where the distribution of any symbol (0 or 1) may depend on an…
The combinatorial basis of entropy by Boltzmann can be written $H= {N}^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy of a system, per unit entity, $N$ is the number of entities and $\mathbb{W}$ is the number of ways in which…
The Bernstein-Sato polynomial is an important invariant of an element or an ideal in a polynomial ring or power series ring of characteristic zero, with interesting connections to various algebraic and topological aspects of the…
We obtain a Bernstein-type inequality for sums of Banach-valued random variables satisfying a weak dependence assumption of general type and under certain smoothness assumptions of the underlying Banach norm. We use this inequality in order…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
Let $X$ be an irreducible shift of finite type (SFT) of positive entropy, and let $B_n(X)$ be its set of words of length $n$. Define a random subset $\omega$ of $B_n(X)$ by independently choosing each word from $B_n(X)$ with some…
In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
Using proof-theoretic methods in the style of proof mining, we give novel computationally effective limit theorems for the convergence of the Cesaro-means of certain sequences of random variables. These results are intimately related to…
We provide a general result for bounding the difference between point probabilities of integer supported distributions and the translated Poisson distribution, a convenient alternative to the discretized normal. We illustrate our theorem in…
We investigate the R\'enyi entropy of independent sums of integer valued random variables through Fourier theoretic means, and give sharp comparisons between the variance and the R\'enyi entropy, for Poisson-Bernoulli variables. As…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
We develop the theory of algorithmic randomness for the space $A^G$ where $A$ is a finite alphabet and $G$ is a computable amenable group. We give an effective version of the Shannon-McMillan-Breiman theorem in this setting. We also extend…
The sum of $n$ {non-independent} Bernoulli random variables could be modeled in several different ways. One of these is the Multiplicative Binomial Distribution (MBD), introduced by Altham (1978) and revised by Lovison (1998). In this work,…
Benford's law is a famous law in statistics which states that the leading digits of random variables in diverse data sets appear not uniformly from 1 to 9; the probability that d (d=1,...,9) appears as a leading digit is given by…
New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input…
Uniform continuity bounds on entropies are generally expressed in terms of a single distance measure between a pair of probability distributions or quantum states, typically, the total variation distance or trace distance. However, if an…
Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in…