Related papers: Motivic Information
Mutual information among three or more dimensions (mu-star = - Q) has been considered as interaction information. However, Krippendorff (2009a, 2009b) has shown that this measure cannot be interpreted as a unique property of the…
Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
It is shown that there is a mapping of the replica approach to disordered systems with finite replica index $n$ on the Tsallis non-extensive statistics, if the average thermodynamic entropy differs from the information entropy for the…
Good generative models should not only synthesize high quality data, but also utilize interpretable representations that aid human understanding of their behavior. However, it is difficult to measure objectively if and to what degree…
The information in an individual finite object (like a binary string) is commonly measured by its Kolmogorov complexity. One can divide that information into two parts: the information accounting for the useful regularity present in the…
We consider the problem of decision-making with side information and unbounded loss functions. Inspired by probably approximately correct learning model, we use a slightly different model that incorporates the notion of side information in…
The relationship between entropy and information is reviewed, taking into account that information is stored in macroscopic degrees of freedom, such as the order parameter in a system exhibiting spontaneous symmetry breaking. It is shown…
Information-theoretic measures have been widely adopted in the design of features for learning and decision problems. Inspired by this, we look at the relationship between i) a weak form of information loss in the Shannon sense and ii) the…
An information measure inspired by Onicescu's information energy and Uffink's information measure (recently discussed by Brukner and Zeilinger) are calculated as functions of the number of particles $N$ for fermionic systems (nuclei and…
This study explores information measures based on extropy, introducing dynamic relative extropy measures for residual and past lifetimes, and investigating their various properties. Furthermore, the study analyzes the relationships between…
Spiking activity from populations of neurons display causal interactions and memory effects. Therefore, they are expected to show some degree of irreversibility in time. Motivated by the spike train statistics, in this paper we build a…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
We present information value, a measure which quantifies the predictability of an utterance relative to a set of plausible alternatives. We introduce a method to obtain interpretable estimates of information value using neural text…
Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
Inspired by works on information transmission through quantum channels, we propose the use of a couple of mutual entropies to quantify the efficiency of continual measurement schemes in extracting information on the measured quantum system.…
The additional information carried by an enlarged filtration and its measurement was studied by several authors. Already Meyer (Sur un theoreme de J. Jacod, 1978) and Yor (Entropie d'une partition, et grossissement initial d'une filtration,…
The tension between qualitative theorizing and quantitative methods is pervasive in the social sciences, and poses a constant challenge to empirical research. But in science studies as an interdisciplinary specialty, there are additional…
Transfer Entropy and Directed Information are information-theoretic measures of the directional dependency between stochastic processes. Following the definitions of Schreiber and Massey in discrete time, we define and evaluate these…