Related papers: A Principled Infotheoretic \phi-like Measure
Complex systems, from the human brain to the global economy, are made of multiple elements that interact in such ways that the behaviour of the `whole' often seems to be more than what is readily explainable in terms of the `sum of the…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
For readers already familiar with Partial Information Decomposition (PID), we show that PID's definition of synergy enables quantifying at least four different notions of irreducibility. First, we show four common notions of "parts" give…
We introduce a hierarchical classification of theories that describe systems with fundamentally limited information content. This property is introduced in an operational way and gives rise to the existence of mutually complementary…
We address three outstanding problems in information theory. Problem one is the definition of a non-negative decomposition of the information conveyed by two or more sources about a target variable into the specific contribution of each…
The concept of information has emerged as a language in its own right, bridging several disciplines that analyze natural phenomena and man-made systems. Integrated information has been introduced as a metric to quantify the amount of…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
Information theory is a powerful framework for quantifying complexity, uncertainty, and dynamical structure in time-series data, with widespread applicability across disciplines such as physics, finance, and neuroscience. However, the…
This overview of integrated information theory (IIT) emphasizes IIT's "consciousness-first" approach to what exists. Consciousness demonstrates to each of us that something exists--experience--and reveals its essential properties--the…
Informational entropy is often identified as physical entropy. This is surprising because the two quantities are differently defined and furthermore the former is a subjective quantity while the latter is an objective one. We describe the…
Partial Information Decomposition (PID) has become one of the most prominent information-theoretic frameworks for describing the structure and quality of information in complex systems. Despite its widespread utility, there exists no unique…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…
The concept of autonomy is fundamental for understanding biological organization and the evolutionary transitions of living systems. Understanding how a system constitutes itself as an individual, cohesive, self-organized entity is a…
Inference and learning are commonly cast in terms of optimisation, yet the fundamental constraints governing uncertainty reduction remain unclear. This work presents a first-principles framework inherent to Bayesian updating, termed…
A novel measure, quantumness of correlations is introduced here for bipartite states, by incorporating the required measurement scheme crucial in defining any such quantity. Quantumness coincides with the previously proposed measures in…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease…