Related papers: Information geometric complexity of a trivariate G…
We study a differential geometric construction, the warped product, on the background geometry for information theory. Divergences, dual structures and symmetric 3-tensor are studied under this construction, and we show that warped product…
In the context of statistical learning, the Information Bottleneck method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description…
Recent studies of networks representing complex systems from the brain to social graphs have revealed their higher-order architecture, which can be described by aggregates of simplexes (triangles, tetrahedrons, and higher cliques). Current…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
Information geometry has offered a way to formally study the efficacy of scientific models by quantifying the impact of model parameters on the predicted effects. However, there has been little formal investigation of causation in this…
Random field models are mathematical structures used in the study of stochastic complex systems. In this paper, we compute the shape operator of Gaussian random field manifolds using the first and second fundamental forms (Fisher…
In information geometry, one of the basic problem is to study the geomet-ric properties of statistical manifold. In this paper, we study the geometricstructure of the generalized normal distribution manifold and show that it has constant…
The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding geometric structure has been demonstrated empirically, rigorous analysis of its impact on the learnability…
Prediction of events is the challenge in many different disciplines, from meteorology to finance; the more this task is difficult, the more a system is {\it complex}. Nevertheless, even according to this restricted definition, a general…
Fisher Information (FI) is a quantity ubiquitously measured in such varied areas like metrology, machine learning, and biological complexity. Mathematically, it represents a lower bound in the variance of unknown parameters that are related…
This article elaborates on entanglement entropy and quantum information theory of geometric flows of (relativistic) Lagrange--Hamilton mechanical systems. A set of basic geometric and quantum mechanics and probability concepts together with…
Information geometry uses the formal tools of differential geometry to describe the space of probability distributions as a Riemannian manifold with an additional dual structure. The formal equivalence of compositional data with discrete…
We use the method of maximum entropy to model physical space as a curved statistical manifold. It is then natural to use information geometry to explain the geometry of space. We find that the resultant information metric does not describe…
Configurational information is generated when three or more sources of variance interact. The variations not only disturb each other relationally, but by selecting upon each other, they are also positioned in a configuration. A…
Information geometry provides a tool to systematically investigate parameter sensitivity of the state of a system. If a physical system is described by a linear combination of eigenstates of a complex (that is, non-Hermitian) Hamiltonian,…
We apply a recently developed measure of multiscale complexity to the Gaussian model consisting of continuous spins with bilinear interactions for a variety of interaction matrix structures. We find two universal behaviors of the complexity…
The emergence of cosmic structure is commonly considered one of the most complex phenomena in Nature. However, this complexity has never been defined nor measured in a quantitative and objective way. In this work we propose a method to…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
We study the effect of the choice of embedding geometry on the entropy of random geometric graph ensembles with soft connection functions. First we show that when the connection range is small, the entropy is dependent only on the dimension…
The entropy and information puzzles arising from black holes cannot be resolved if quantum gravity effects remain confined to a microscopic scale. We use concrete computations in nonperturbative string theory to argue for three kinds of…