Related papers: The Basics of Information Geometry
Geometry constitutes a core set of intuitions present in all humans, regardless of their language or schooling [1]. Could brain's built in machinery for processing geometric information take part in uncertainty representation? For decades…
High dimensional data can have a surprising property: pairs of data points may be easily separated from each other, or even from arbitrary subsets, with high probability using just simple linear classifiers. However, this is more of a rule…
In this paper and a companion paper, we show how the framework of information geometry, a geometry of discrete probability distributions, can form the basis of a derivation of the quantum formalism. The derivation rests upon a few…
The curse of dimensionality is a common phenomenon which affects analysis of datasets characterized by large numbers of variables associated with each point. Problematic scenarios of this type frequently arise in classification algorithms…
It is known that statistical model selection as well as identification of dynamical equations from available data are both very challenging tasks. Physical systems behave according to their underlying dynamical equations which, in turn, can…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
We investigate the geometrical structure of probabilistic generative dimensionality reduction models using the tools of Riemannian geometry. We explicitly define a distribution over the natural metric given by the models. We provide the…
We describe the fundamental constructions and properties of determinantal probability measures and point processes, giving streamlined proofs. We illustrate these with some important examples. We pose several general questions and…
Several mathematical ideas have been investigated for Quantitative Information Flow. Information theory, probability, guessability are the main ideas in most proposals. They aim to quantify how much information is leaked, how likely is to…
Message identification (M-I) divergence is an important measure of the information distance between probability distributions, similar to Kullback-Leibler (K-L) and Renyi divergence. In fact, M-I divergence with a variable parameter can…
We describe the geometric notion of distribution in synthetic terms, utilizing the notion of "first neighbourhood of the diagonal" from algebraic geometry. We characterize involutive distributions in combinatorial terms.
Upon a consistent topological statistical theory the application of structural statistics requires a quantification of the proximity structure of model spaces. An important tool to study these structures are Pseudo-Riemannian metrices,…
Network analysis has a crucial need for tools to compare networks and assess the significance of differences between networks. We propose a principled statistical approach to network comparison that approximates networks as probability…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
We present and study approximate notions of dimensional and margin complexity, which correspond to the minimal dimension or norm of an embedding required to approximate, rather then exactly represent, a given hypothesis class. We show that…
Starting from considerations about meaning and subsequent use of asymmetric uncertainty intervals of experimental results, we review the issue of uncertainty propagation. We show that, using a probabilistic approach (the so-called Bayesian…
Traffic is constrained by the information involved in locating the receiver and the physical distance between sender and receiver. We here focus on the former, and investigate traffic in the perspective of information handling. We re-plot…
This paper considers the problem of defining a measure of redundant information that quantifies how much common information two or more random variables specify about a target random variable. We discussed desired properties of such a…
This talk reviews some mathematical and physical ideas related to the notion of dimension. After a brief historical introduction, various modern constructions from fractal geometry, noncommutative geometry, and theoretical physics are…
Metrics for rigorously defining a distance between two events have been used to study the properties of the dataspace manifold of particle collider physics. The probability distribution of pairwise distances on this dataspace is unique with…