Related papers: Information geometry and sufficient statistics
Information geometry is used to quantify the amount of information integration within multiple terminals of a causal dynamical system. Integrated information quantifies how much information is lost when a system is split into parts and…
The high-dimensional parameter space of deep neural networks -- the neuromanifold -- is endowed with a unique metric tensor defined by the Fisher information. Reliable and scalable computation of this metric tensor is valuable for theorists…
Contrast functions play a fundamental role in information geometry, providing a means for generating the geometric structures of a statistical manifold: a pseudo-Riemannian metric and a pair of torsion-free conjugate affine connections.…
We develop a family of infinite-dimensional Banach manifolds of measures on an abstract measurable space, employing charts that are "balanced" between the density and log-density functions. The manifolds, $(\tilde{M}_{\lambda},\lambda\in…
We present a novel geometric perspective on the latent space of diffusion models. We first show that the standard pullback approach, utilizing the deterministic probability flow ODE decoder, is fundamentally flawed. It provably forces…
This paper presents new geometric aspects of the behaviors of solutions to the porous medium equation (PME) and its associated equation. First we discuss the Legendre structure with information geometry on the manifold of generalized…
We discuss the recently proposed description of Kuramoto model in terms of hyperbolic space and relate it to the information geometry. In particular the dynamical equation in Kuramoto all-to-all model is identified with the gradient flow of…
We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originally introduced in theoretical ecology to study the diversity…
Following Thurston's geometrisation picture in dimension three, we study geometric manifolds in a more general setting in arbitrary dimensions, with respect to the following problems: (i) The existence of maps of non-zero degree (domination…
Natural gradient is an advanced optimization method based on information geometry, where the Fisher metric plays a crucial role. Its quantum counterpart, known as quantum natural gradient (QNG), employs the symmetric logarithmic derivative…
We derive bounds for the Orlicz norm of the deviation of a random variable defined on $\mathbb{R}^n$ from its Gaussian mean value. The random variables are assumed to be smooth and the bound itself depends on the Orlicz norm of the…
We discuss a relationship between information geometry and the Glansdorff-Prigogine criterion for stability. For the linear master equation, we found a relation between the line element and the excess entropy production rate. This relation…
Structured optical beams possess rich spatial features that are commonly characterized using entropic measures of field complexity. However, such measures do not directly quantify the operational usefulness of optical structure for…
Quantum Fisher information places the fundamental limit to the accuracy of estimating an unknown parameter. Here we shall provide the quantum Fisher information an operational meaning: a mixed state can be so prepared that a given…
Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular, the phenomenon that the marginal distribution of the data…
We extend the concept of Wigner-Yanase-Dyson skew information to something we call ``metric adjusted skew information'' (of a state with respect to a conserved observable). This ``skew information'' is intended to be a non-negative quantity…
In the world of generalized entropies---which, for example, play a role in physical systems with sub- and super-exponential phasespace growth per degree of freedom---there are two ways for implementing constraints in the maximum entropy…
In this paper I discuss the relation between the concept of the Fisher metric and the concept of differentiability of a family of probability measures. I compare the concepts of smooth statistical manifolds, differentiable families of…
We construct and analyze a thermodynamic extension of the recently proposed information geometric regularization of Cao and Sch\"afer. The construction extends their shock-mitigating Hessian metric geometry using the Shannon entropy to…
Hierarchical parametric models consisting of observable and latent variables are widely used for unsupervised learning tasks. For example, a mixture model is a representative hierarchical model for clustering. From the statistical point of…