Related papers: Summary of Information Theoretic Quantities
One of the key aspects of Shannon's theory is that it provides guidance for designing the most efficient systems, such as minimizing errors and clarifying the limits of coding. Such theories have made great developments in the 50 years…
How much information is transmitted when animals use emotions to communicate? It is clear that emotions are used as communication systems in humans and other species. The quantitative theory of emotion information presented here is based on…
A mathematical framework for information-theoretic analysis is established, with a new viewpoint of describing transmitted messages and communication channels by the nonlinear expectation theory, beyond the framework of classical…
Information flow provides a natural measure for the causal interaction between dynamical events. This study extends our previous rigorous formalism of componentwise information flow to the bulk information flow between two complex…
Turbulence theory is usually concerned with the statistical moments of the velocity or its fluctuations. One could also analyze the implicit probability distributions. This is the purview of information theory. Here we use information…
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
Information is the basic concept of information theory. However, there is no definition of this concept that can encompass all uses of the term information in information theories and beyond. Many question a possibility of such a…
The field of Information Theory is founded on Claude Shannon's seminal ideas relating to entropy. Nevertheless, his well-known avoidance of meaning (Shannon, 1948) still persists to this day, so that Information Theory remains poorly…
It has been suggested, on the one hand, that quantum states are just states of knowledge; and, on the other, that quantum theory is merely a theory of correlations. These suggestions are confronted with problems about the nature of…
Information theory is a branch of probability and statistics involving the analysis of communications. Information theory enables us to analyze and quantify the information content of predictions made in the context of plant disease…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…
In an effort to develop the foundations for a non-stochastic theory of information, the notion of $\delta$-mutual information between uncertain variables is introduced as a generalization of Nair's non-stochastic information functional.…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Most psychophysical experiments discard half the data collected. Specifically, experiments discard reaction time data, and use binary responses (e.g. yes/no) to measure performance. Here, Shannon's information theory is used to define…
We define the concept of dependence among multiple variables using maximum entropy techniques and introduce a graphical notation to denote the dependencies. Direct inference of information theoretic quantities from data uncovers…
Originally conceived as a theory of consciousness, integrated information theory (IIT) provides a theoretical framework intended to characterize the compositional causal information that a system, in its current state, specifies about…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
Information must take up space, must weigh, and its flux must be limited. Quantum limits on communication and information storage leading to these conclusions are here described. Quantum channel capacity theory is reviewed for both steady…
We propose a novel tensor-based formalism for inferring causal structures from time series. An information theoretical analysis of transfer entropy, shows that transfer entropy results from transmission of information over a set of…
Integrated Information Theory (IIT) has emerged as one of the leading research lines in computational neuroscience to provide a mechanistic and mathematically well-defined description of the neural correlates of consciousness. Integrated…