Related papers: Entropy and a generalisation of `Poincare's Observ…

A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…

Kullback-Leibler relative-entropy, in cases involving distributions resulting from relative-entropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras' theorem. And…

We generalise the known fact that for binomial $X_{n,k} \sim \mathrm{Bin}(n, k/n)$ one has $\inf_{k>1,n} \mathrm{P}(X_{n,k} \geq k) \geq \lim_{k \to 1+}\mathrm{P}(X_{2,k} \geq k) = 1/4$ to cover probabilities of exceeding a constant shift…

Observational entropy -- a quantity that unifies Boltzmann's entropy, Gibbs' entropy, von Neumann's macroscopic entropy, and the diagonal entropy -- has recently been argued to play a key role in a modern formulation of statistical…

We determine the joint limiting distribution of adjacent spacings around a central, intermediate, or an extreme order statistic $X_{k:n}$ of a random sample of size $n$ from a continuous distribution $F$. For central and intermediate cases,…

An explicit bound is given for the Kolmogorov distance between a mixture of normal distributions and a normal distribution with properly chosen parameter values. A random variable X has a mixture of normal distributions if its conditional…

We define the quantile set of order $\alpha \in \left[ 1/2,1\right) $ associated to a law $P$ on $\mathbb{R}^{d}$ to be the collection of its directional quantiles seen from an observer $O\in \mathbb{R}^{d}$. Under minimal assumptions these…

We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…

We investigate the $m$-relative entropy, which stems from the Bregman divergence, on weighted Riemannian and Finsler manifolds. We prove that the displacement $K$-convexity of the $m$-relative entropy is equivalent to the combination of the…

We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…

We prove a Donsker and a Glivenko--Cantelli theorem for sequences of random discrete measures generalizing empirical measures. Those two results hold under standard conditions upon bracketing numbers of the indexing class of functions. As a…

Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…

For random systems of $K$ polynomials in $N + 1$ real variables which include the models of Kostlan (1987) and Shub and Smale (1993), we prove that the number of zeros on the unit sphere for $K = N$ or the Hausdorff measure of the zero set…

We consider an $N$ by $N$ real symmetric random matrix $X=(x_{ij})$ where $\mathbb{E}x_{ij}x_{kl}=\xi_{ijkl}$. Under the assumption that $(\xi_{ijkl})$ is the discretization of a piecewise Lipschitz function and that the correlation is…

The coordinates along any fixed direction(s), of points on the sphere $S^{n-1}(\sqrt{n})$, roughly follow a standard Gaussian distribution as $n$ approaches infinity. We revisit this classical result from a nonstandard analysis perspective,…

Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…

We find, in an intrinsic form, a generalized Rankine-Hugoniot condition with respect to an entropy density that allows to give the proper interpretation to a formula of Vol'pert reducing the entropy condition on a function with bounded…

We show that almost any one-dimensional projection of a suitably scaled random walk on a hypercube, inscribed in a hypersphere, converges weakly to an Ornstein-Uhlenbeck process as the dimension of the sphere tends to infinity. We also…

We consider diffraction at random point scatterers on general discrete point sets in $\R^\nu$, restricted to a finite volume. We allow for random amplitudes and random dislocations of the scatterers. We investigate the speed of convergence…

Quantum information-theoretic approach has been identified as a way to understand the foundations of quantum mechanics as early as 1950 due to Shannon. However there hasn't been enough advancement or rigorous development of the subject. In…