Related papers: GAIT: A Geometric Approach to Information Theory
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
Network or graph structures are ubiquitous in the study of complex systems. Often, we are interested in complexity trends of these system as it evolves under some dynamic. An example might be looking at the complexity of a food web as…
High-order phenomena play crucial roles in many systems of interest, but their analysis is often highly nontrivial. There is a rich literature providing a number of alternative information-theoretic quantities capturing high-order…
Geometrical methods in quantum information are very promising for both providing technical tools and intuition into difficult control or optimization problems. Moreover, they are of fundamental importance in connecting pure geometrical…
In many high-impact applications, it is important to ensure the quality of output of a machine learning algorithm as well as its reliability in comparison with the complexity of the algorithm used. In this paper, we have initiated a…
Observational entropy provides a general notion of quantum entropy that appropriately interpolates between Boltzmann's and Gibbs' entropies, and has recently been argued to provide a useful measure of out-of-equilibrium thermodynamic…
We give a notion of entropy for general gemetric structures, which generalizes well-known notions of topological entropy of vector fields and geometric entropy of foliations, and which can also be applied to singular objects, e.g. singular…
A geometric entropy is defined as the Riemannian volume of the parameter space of a statistical manifold associated with a given network. As such it can be a good candidate for measuring networks complexity. Here we investigate its ability…
We introduce a novel geometric approach to characterize entanglement relations in large quantum systems. Our approach is inspired by Schumacher's singlet state triangle inequality, which used an entropic-based distance to capture the…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Entropy is being used in physics, mathematics, informatics and in related areas to describe equilibration, dissipation, maximal probability states and optimal compression of information. The Gini index on the other hand is an established…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second…
The manifold of empirical mean values of statistical data ad infinitum has a geometric shape that depends on the probability measure that governs the generating model. Large deviation theory produces entropy functions that depend on both…
Complex models in physics, biology, economics, and engineering are often sloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades…
We attempt to describe geometry in terms of informational quantities for the universe considered as a finite ensemble of correlated quantum particles. As the main dynamical principle, we use the conservation of the sum of all kinds of…
Systems with a long-term stationary state that possess as a spatio-temporally fluctuation quantity $\beta$ can be described by a superposition of several statistics, a "super statistics". We consider first, the Gamma, log-normal and…
In this tutorial we review the essential arguments behing entropic inference. We focus on the epistemological notion of information and its relation to the Bayesian beliefs of rational agents. The problem of updating from a prior to a…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
Kolmogorov argued that the concept of information exists also in problems with no underlying stochastic model (as Shannon's information representation) for instance, the information contained in an algorithm or in the genome. He introduced…