Related papers: Function-space regularized R\'enyi divergences
We consider a two-parameter family of R\'enyi relative entropies $D_{\alpha,z}(\rho||\sigma)$ that are quantum generalisations of the classical R\'enyi divergence $D_{\alpha}(p||q)$. This family includes many known relative entropies (or…
Neural Processes (NPs) are deep probabilistic models that represent stochastic processes by conditioning their prior distributions on a set of context points. Despite their advantages in uncertainty estimation for complex distributions, NPs…
Sparse high-dimensional linear regression is a central problem in statistics, where the goal is often variable selection and/or coefficient estimation. We propose a mean-field variational Bayes approximation for sparse regression with…
This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as…
Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential…
In the field of statistics, many kind of divergence functions have been studied as an amount which measures the discrepancy between two probability distributions. In the differential geometrical approach in statistics (information…
The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the…
This paper revisits Deep Mutual Learning (DML), a simple yet effective computing paradigm. We propose using R\'{e}nyi divergence instead of the KL divergence, which is more flexible and tunable, to improve vanilla DML. This modification is…
Fawzi and Fawzi recently defined the sharp R\'enyi divergence, $D_\alpha^\#$, for $\alpha \in (1, \infty)$, as an additional quantum R\'enyi divergence with nice mathematical properties and applications in quantum channel discrimination and…
We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram…
We consider the problem of learning functions within the $\mathcal{F}_{p,\pi}$ and Barron spaces, which play crucial roles in understanding random feature models (RFMs), two-layer neural networks, as well as kernel methods. Leveraging tools…
We study the $\alpha$-$z$-R\'enyi divergences $D_{\alpha,z}(\psi\|\varphi)$ where $\alpha,z>0$ ($\alpha\ne1$) for normal positive functionals $\psi,\varphi$ on general von Neumann algebras, introduced in [S.~Kato and Y.~Ueda,…
We provide an operational interpretation of the multivariate R\'enyi divergence in terms of economic-theoretic tasks based on betting, risk aversion, and multiple lotteries. We show that the multivariate R\'enyi divergence…
Complex statistical models are often built by combining multiple submodels, called modules. Here we consider modular inference where the modules contain both parametric and nonparametric components. In such cases, standard Bayesian…
We derive the information geometry induced by the statistical R\'enyi divergence, namely its metric tensor, its dual parametrized connections, as well as its dual Laplacians. Based on these results, we demonstrate that the R\'enyi-geometry,…
We study the asymptotic consistency properties of $\alpha$-R\'enyi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the…
A discrete system's heterogeneity is measured by the R\'enyi heterogeneity family of indices (also known as Hill numbers or Hannah--Kay indices), whose units are {the numbers equivalent}. Unfortunately, numbers equivalent heterogeneity…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
This paper develops systematic approaches to obtain $f$-divergence inequalities, dealing with pairs of probability measures defined on arbitrary alphabets. Functional domination is one such approach, where special emphasis is placed on…
In this work we introduce a family of transformations, named \textit{divergence transformations}, interpolating between any pair of probability density functions sharing the same support. We prove the remarkable property that the whole…