Related papers: A numerical approximation method for the Fisher-Ra…

Estimating the Kullback--Leibler (KL) divergence between language models has many applications, e.g., reinforcement learning from human feedback (RLHF), interpretability, and knowledge distillation. However, computing the exact KL…

Analyzing high-dimensional data with manifold learning algorithms often requires searching for the nearest neighbors of all observations. This presents a computational bottleneck in statistical manifold learning when observations of…

This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The…

We propose a new embedding method for a single vector and for a pair of vectors. This embedding method enables: a) efficient classification and regression of functions of single vectors; b) efficient approximation of distance functions; and…

We study the geometry of probability distributions with respect to a generalized family of Csisz\'ar $f$-divergences. A member of this family is the relative $\alpha$-entropy which is also a R\'enyi analog of relative entropy in information…

We prove that the $f$-divergences between univariate Cauchy distributions are all symmetric, and can be expressed as strictly increasing scalar functions of the symmetric chi-squared divergence. We report the corresponding scalar functions…

Probability distributions in Stiefel manifold such as the von-Mises Fisher and Bingham distributions find diverse applications in signal processing and other applied sciences. Use of these statistical models in practice is complicated by…

We show that the predicted probability distributions for any $N$-parameter statistical model taking the form of an exponential family can be explicitly and analytically embedded isometrically in a $N{+}N$-dimensional Minkowski space. That…

Local geometric information, i.e. normal and distribution of points, is crucial for LiDAR-based simultaneous localization and mapping (SLAM) because it provides constraints for data association, which further determines the direction of…

Starting with a similarity function between objects, it is possible to define a distance metric on pairs of objects, and more generally on probability distributions over them. These distance metrics have a deep basis in functional analysis,…

The Fisher-Snedecor $\mathcal{F}$ distribution has been recently proposed as a more accurate and mathematically tractable composite fading model than traditional established models in some practical cases. In this paper, we firstly derive…

Information divergences allow one to assess how close two distributions are from each other. Among the large panel of available measures, a special attention has been paid to convex $\varphi$-divergences, such as Kullback-Leibler,…

Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty…

Reflected diffusions in polyhedral domains are commonly used as approximate models for stochastic processing networks in heavy traffic. Stationary distributions of such models give useful information on the steady state performance of the…

$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler…

We design and implement a novel algorithm for computing a multilevel Monte Carlo (MLMC) estimator of the cumulative distribution function of a quantity of interest in problems with random input parameters or initial conditions. Our approach…

We deal with the task of sampling from an unnormalized Boltzmann density $\rho_D$ by learning a Boltzmann curve given by energies $f_t$ starting in a simple density $\rho_Z$. First, we examine conditions under which Fisher-Rao flows are…

This is a review of the ideas behind the Fisher--Rao metric on classical probability distributions, and how they generalize to metrics on density matrices. As is well known, the unique Fisher--Rao metric then becomes a large family of…

Idealized probability distributions, such as normal or other curves, lie at the root of confirmatory statistical tests. But how well do people understand these idealized curves? In practical terms, does the human visual system allow us to…

This paper focuses on Sequential Monte Carlo approximations of smoothing distributions in conditionally linear and Gaussian state spaces. To reduce Monte Carlo variance of smoothers, it is typical in these models to use…