Related papers: Information Theory for Expectation Measures
We present a problem relating measurements and information theory in spin foam models. In the three dimensional case of quantum gravity we can compute probabilities of spin network graphs and study the behaviour of the Shannon entropy…
We survey the prospects for an Information Dynamics which can serve as the basis for a fundamental theory of information, incorporating qualitative and structural as well as quantitative aspects. We motivate our discussion with some basic…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
Logical probability theory was developed as a quantitative measure based on Boole's logic of subsets. But information theory was developed into a mature theory by Claude Shannon with no such connection to logic. A recent development in…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
Quantum information-theoretic approach has been identified as a way to understand the foundations of quantum mechanics as early as 1950 due to Shannon. However there hasn't been enough advancement or rigorous development of the subject. In…
Information theory gives rise to a novel method for causal skeleton discovery by expressing associations between variables as tensors. This tensor-based approach reduces the dimensionality of the data needed to test for conditional…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and robust framework to study such behaviours, with several…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
This paper is generally concerned with understanding how the uncertainty principle arises in formulations of quantum mechanics, such as the decoherent histories approach, whose central goal is the assignment of probabilities to histories.…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
We study an information-theoretic measure of uncertainty for quantum systems. It is the Shannon information $I$ of the phase space probability distribution $\la z | \rho | z \ra $, where $|z \ra $ are coherent states, and $\rho$ is the…
A general theory for computing information transfers in nonlinear stochastic systems driven by deterministic forcing and additive and/or multiplicative noises, is presented. It extends the Liang-Kleeman framework of causality inference to…
Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and…
The information-theoretic representation of quantum systems, which complements the familiar energy description of the density-functional and wave-function-based theories, is here discussed. According to it, the internal disorder of the…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the…