Related papers: The Basics of Information Geometry
Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information,…
In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information manifolds in information…
This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…
Probabilistic graphical models allow us to encode a large probability distribution as a composition of smaller ones. It is oftentimes the case that we are interested in incorporating in the model the idea that some of these smaller…
This paper is about Information Geometry, a relatively new subject within mathematical statistics that attempts to study the problem of inference by using tools from modern differential geometry. This paper provides an overview of some of…
Can the spatial distance between two identical particles be explained in terms of the extent that one can be distinguished from the other? Is the geometry of space a macroscopic manifestation of an underlying microscopic statistical…
This paper is a strongly geometrical approach to the Fisher distance, which is a measure of dissimilarity between two probability distribution functions. The Fisher distance, as well as other divergence measures, are also used in many…
Exploiting the geometric nature of statistical divergences, we devise a way to define associated induced uncertainty measures for discrete and finite probability distributions. We also report new uncertainty measures and discuss their…
Information geometry is used to quantify the amount of information integration within multiple terminals of a causal dynamical system. Integrated information quantifies how much information is lost when a system is split into parts and…
This document reviews the definition of the kernel distance, providing a gentle introduction tailored to a reader with background in theoretical computer science, but limited exposure to technology more common to machine learning,…
We present a definition of the distance between probability distributions. Our definition is based on the $L_1$ norm on space of probability measures. We compare our distance with the well-known Kullback-Leibler divergence and with the…
The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximated by binomial distributions or Poisson distributions. In…
In pattern recognition, learning, and data mining one obtains information from information-carrying objects. This involves an objective definition of the information in a single object, the information to go from one object to another…
The quantale of distance distributions is of fundamental importance for understanding probabilistic metric spaces as enriched categories. Motivated by the categorical interpretation of partial metric spaces, we are led to investigate the…
Various kinds of data are routinely represented as discrete probability distributions. Examples include text documents summarized by histograms of word occurrences and images represented as histograms of oriented gradients. Viewing a…
As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…
In this work we: (1) review likelihood-based inference for parameter estimation and the construction of confidence regions; and, (2) explore the use of techniques from information geometry, including geodesic curves and Riemann scalar…
We give an overview of different approaches to measuring the similarity of, or the distance between, two graphs, highlighting connections between these approaches. We also discuss the complexity of computing the distances.
A flat membrane with given shape is displayed; two points in the membrane are randomly selected; the probability that the separation between the points have a specified value is sought. A simple method to evaluate the probability density is…
This introductory text arises from a lecture given in G\"oteborg, Sweden, given by the first author and is intended for undergraduate students, as well as for any mathematically inclined reader wishing to explore a synthesis of ideas…