Related papers: A Variational Characterization of R\'enyi Divergen…
We propose an extension of the classical R\'enyi divergences to quantum states through an optimization over probability distributions induced by restricted sets of measurements. In particular, we define the notion of locally-measured…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. We consider shift-invariant probability measures satisfying some…
We extend the duality between exponential integrals and relative entropy to a variational formula for exponential integrals involving the Renyi divergence. This formula characterizes the dependence of risk-sensitive functionals and related…
We present a variational characterization for the R\'{e}nyi divergence of order infinity. Our characterization is related to guessing: the objective functional is a ratio of maximal expected values of a gain function applied to the…
We derive a new variational formula for the R\'enyi family of divergences, $R_\alpha(Q\|P)$, between probability measures $Q$ and $P$. Our result generalizes the classical Donsker-Varadhan variational formula for the Kullback-Leibler…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
In this article, we introduce a notion of relative mean metric dimension with potential for a factor map $\pi: (X,d, T)\to (Y, S)$ between two topological dynamical systems. To link it with ergodic theory, we establish four variational…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
We propose a new family of regularized R\'enyi divergences parametrized not only by the order $\alpha$ but also by a variational function space. These new objects are defined by taking the infimal convolution of the standard R\'enyi…
This paper introduces the variational R\'enyi bound (VR) that extends traditional variational inference to R\'enyi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth…
The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi…
A new sharp inequality featuring the differential R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross-entropy of a pair of probability density functions is established. The equality is reached when one of the probability density…
Let $A$ be a transition probability kernel on a finite state space $\Delta^o =\{1, \ldots , d\}$ such that $A(x,y)>0$ for all $x,y \in \Delta^o$. Consider a reinforced chain given as a sequence $\{X_n, \; n \in \mathbb{N}_0\}$ of…
We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…
The Bou\'e-Dupuis variational formula gives a representation for log Laplace transforms of bounded measurable functions of a finite dimensional Brownian motion on a compact time interval as an infimum of a suitable cost over a collection of…
We resume the results from \cite{Vershik FA} on the classification of measurable functions in several variables, with some minor corrections of purely technical nature, and give a partial solution to the characterization problem of…
In this paper, we examine the existence of the R\'enyi divergence between two time invariant general hidden Markov models with arbitrary positive initial distributions. By making use of a Markov chain representation of the probability…