Related papers: On Reverse Pinsker Inequalities
Let $(X,\mu)$ be a standard probability space. An automorphism $T$ of $(X,\mu)$ has the weak Pinsker property if for every $\varepsilon > 0$ it has a splitting into a direct product of a Bernoulli shift and an automorphism of entropy less…
We present a series of new and more favorable margin-based learning guarantees that depend on the empirical margin loss of a predictor. We give two types of learning bounds, both distribution-dependent and valid for general families, in…
We study the continuity property of the generalized entropy as a function of the underlying probability distribution, defined with an action space and a loss function, and use this property to answer the basic questions in statistical…
In this paper, we derive new probability bounds for Chebyshev's inequality if the supremum of the probability density function is known. This result holds for one-dimensional or multivariate continuous probability distributions with finite…
We show sharp bounds for probabilities of large deviations for sums of independent random variables satisfying Bernstein's condition. One such bound is very close to the tail of the standard Gaussian law in certain case; other bounds…
We consider a Markov chain on $\mathbb{R}^d$ with invariant measure $\mu$. We are interested in the rate of convergence of the empirical measures towards the invariant measure with respect to various dual distances, including in particular…
The second law of thermodynamics states that entropy increases (or does not change) by time in an isolated system. As microscopic physical laws are reversible, the origin of irreversibility is not straightforward. Although the outcome of a…
We establish analogs of Cheeger's inequality for probability measures with heavy tails. As one of the principal applications, suppose $\lambda > 3$ and define the (Pareto) probability measure $\mu_{\lambda}$ on $[1,\infty)$ by…
We give exponential upper bounds for $P(S \le k)$, in particular $P(S=0)$, where $S$ is a sum of indicator random variables that are positively associated. These bounds allow, in particular, a comparison with the independent case. We give…
Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}$), arise as redundancies under…
This paper studies the complexity of estimating Renyi divergences of discrete distributions: $p$ observed from samples and the baseline distribution $q$ known \emph{a priori}. Extending the results of Acharya et al. (SODA'15) on estimating…
Elaborating on a previous work by Marolf et al, we relate some exact results in quantum field theory and statistical mechanics to the Bekenstein universal bound on entropy. Specifically, we consider the relative entropy between the vacuum…
We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in…
We extend almost everywhere convergence in Wiener-Wintner ergodic theorem for $\sigma$-finite measure to a generally stronger almost uniform convergence and present a larger, universal, space for which this convergence holds. We then extend…
Consider a sphere of radius root(n) in n dimensions, and consider X, a random variable uniformly distributed on its surface. Poincare's Observation states that for large n, the distribution of the first k coordinates of X is close in total…
We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
We introduce a relative variant of information loss to characterize the behavior of deterministic input-output systems. We show that the relative loss is closely related to Renyi's information dimension. We provide an upper bound for…
The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures,…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…