Related papers: Approximation and bounding techniques for the Fish…
Recently proposed methods in data subset selection, that is active learning and active sampling, use Fisher information, Hessians, similarity matrices based on gradients, and gradient lengths to estimate how informative data is for a…
The Wasserstein distance is a distance between two probability distributions and has recently gained increasing popularity in statistics and machine learning, owing to its attractive properties. One important approach to extending this…
Uncertainty propagation and filtering can be interpreted as gradient flows with respect to suitable metrics in the infinite dimensional manifold of probability density functions. Such a viewpoint has been put forth in recent literature, and…
This work presents an explicit description of the Fisher-Rao Riemannian metric on the Hilbert manifold of equivalent centered Gaussian measures on an infinite-dimensional Hilbert space. We show that the corresponding quantities from the…
I define a natural measure of the complexity of a parametric distribution relative to a given true distribution called the {\it razor} of a model family. The Minimum Description Length principle (MDL) and Bayesian inference are shown to…
Laplace's method approximates a target density with a Gaussian distribution at its mode. It is computationally efficient and asymptotically exact for Bayesian inference due to the Bernstein-von Mises theorem, but for complex targets and…
This paper introduces a comprehensive framework to adjust a discrete test statistic for improving its hypothesis testing procedure. The adjustment minimizes the Wasserstein distance to a null-approximating continuous distribution, tackling…
This paper presents a novel method for analyzing the latent space geometry of generative models, including statistical physics models and diffusion models, by reconstructing the Fisher information metric. The method approximates the…
In information geometry, statistical models are considered as differentiable manifolds, where each probability distribution represents a unique point on the manifold. A Riemannian metric can be systematically obtained from a divergence…
Supervised dimensionality reduction maps labeled data into a low-dimensional feature space while preserving class discriminability. A common approach is to maximize a statistical measure of dissimilarity between classes in the feature…
A new Riemannian geometry for the Compound Gaussian distribution is proposed. In particular, the Fisher information metric is obtained, along with corresponding geodesics and distance function. This new geometry is applied on a change…
Imaging systems are represented as linear operators, and their singular value spectra describe the structure recoverable at the operator level. Building on an operator-based information-theoretic framework, this paper introduces a minimal…
Many statistical models require an estimation of unknown (co)-variance parameter(s) in a model. The estimation usually obtained by maximizing a log-likelihood which involves log determinant terms. In principle, one requires the…
We present the cosmological distance errors achievable using the baryon acoustic oscillations as a standard ruler. We begin from a Fisher matrix formalism that is upgraded from Seo & Eisenstein (2003). We isolate the information from the…
We investigate the connection between the time-evolution of averages of stochastic quantities and the Fisher information and its induced statistical length. As a consequence of the Cramer-Rao bound, we find that the rate of change of the…
Modern machine learning relies on a collection of empirically successful but theoretically heterogeneous regularization techniques, such as weight decay, dropout, and exponential moving averages. At the same time, the rapidly increasing…
In this paper, a distance between the Gaussian Mixture Models(GMMs) is obtained based on an embedding of the K-component Gaussian Mixture Model into the manifold of the symmetric positive definite matrices. Proof of embedding of K-component…
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability…
In this article, we present recent developments of information geometry, namely, geometry of the Fisher metric, dualistic structures and divergences on the space of probability measures, particularly the theory of geodesics of the Fisher…
We consider the problems of clustering, classification, and visualization of high-dimensional data when no straightforward Euclidean representation exists. Typically, these tasks are performed by first reducing the high-dimensional data to…