Related papers: GAIT: A Geometric Approach to Information Theory
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
Entropy is a famous and well established concept in physics and engineering that can be used for explanation of basic fundamentals as well it finds applications in several areas, from quantum physics to astronomy, from network communication…
We introduce a novel generalization of entropy and conditional entropy from which most definitions from the literature can be derived as particular cases. Within this general framework, we investigate the problem of designing…
Information geometry and inductive inference methods can be used to model dynamical systems in terms of their probabilistic description on curved statistical manifolds. In this article, we present a formal conceptual reexamination of the…
Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish…
This thesis investigates the connection between quantum theory, thermodynamics and information theory. Theories with structure similar to that of quantum theory are considered, mathematically described by the framework of "Generalized…
This book brings new mathematical rigour to the ongoing vigorous debate on how to quantify biological diversity. The question "what is diversity?" has surprising mathematical depth, and breadth too: this book involves parts of mathematics…
The capacity to integrate information is a prominent feature of biological and cognitive systems. Integrated Information Theory (IIT) provides a mathematical approach to quantify the level of integration in a system, yet its computational…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and…
We use the method of maximum entropy to model physical space as a curved statistical manifold. It is then natural to use information geometry to explain the geometry of space. We find that the resultant information metric does not describe…
In this work, we propose a new similarity index for images considering the entropy function and group theory. This index considers an algebraic group of images, it is defined by an inner law that provides a novel approach for the…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…
This article is a brief guide to the field of algorithmic information theory (AIT), its underlying philosophy, and the most important concepts. AIT arises by mixing information theory and computation theory to obtain an objective and…
The participation coefficient is a widely used metric of the diversity of a node's connections with respect to a modular partition of a network. An information-theoretic formulation of this concept of connection diversity, referred to here…
A framework integrating information theory and network science is proposed, giving rise to a potentially new area. By incorporating and integrating concepts such as complexity, coding, topological projections and network dynamics, the…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the…