Related papers: Summary of Information Theoretic Quantities
Quantum mechanics, information theory, and relativity theory are the basic foundations of theoretical physics. The acquisition of information from a quantum system is the interface of classical and quantum physics. Essential tools for its…
We present a theory for modeling the structure of information and a language (Quanta) expressing the theory. Unlike Shannon's information theory, which focuses on the amount of information in an information system, we focus on the structure…
A cornerstone of our understanding of both biological and artificial neural networks is that they store information in the strengths of synaptic connections among the neurons. However, in contrast to the well-established theory for…
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final weights and how much it informs the function computed by the…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
We propose an information-theoretic framework for analyzing control systems based on the close relationship of controllers to communication channels. A communication channel takes an input state and transforms it into an output state. A…
Information theory establishes the ultimate limits on performance for noisy communication systems [Shannon48]. An accurate model of a physical communication device must include quantum effects, but typically including these makes the theory…
Classical and quantum information theory are simply explained. To be more specific it is clarified why Shannon entropy is used as measure of classical information and after a brief review of quantum mechanics it is possible to demonstrate…
In this article a notion of information is presented which stresses the contextuality of quantum objects and their measurement. Mathematically this is reached by a quantification of the quantum mechanical surplus knowledge which has been…
We propose an information-theoretic framework to measure narratives, providing a formalism to understand pivotal moments, cliffhangers, and plot twists. This approach offers creatives and AI researchers tools to analyse and benchmark human-…
In this paper we give a self contained introduction to the conceptional and mathematical foundations of quantum information theory. In the first part we introduce the basic notions like entanglement, channels, teleportation etc. and their…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
Brain activity differs vastly between sleep, cognitive tasks, and action. Information theory is an appropriate concept to analytically quantify these brain states. Based on neurophysiological recordings, this concept can handle complex data…
In this work we build a theoretical framework for the transport of information in quantum systems. This is a framework aimed at describing how out of equilibrium open quantum systems move information around their state space, using an…
A practical notation can convey valuable intuitions and concisely express new ideas. Information theory is of importance to machine learning, but the notation for information-theoretic quantities is sometimes opaque. We propose a practical…
We highlight the underlying category-theoretic structure of measures of information flow. We present an axiomatic framework in which communication systems are represented as morphisms, and information flow is characterized by its behavior…
We give a non-technical introduction of the basic concepts of Quantum Information Theory along the distinction between possible and impossible machines. We then proceed to describe the mathematical framework of Quantum Information Theory.…
Shannon's Capacity Theorem is the main concept behind the Theory of Communication. It says that if the amount of information contained in a signal is smaller than the channel capacity of a physical media of communication, it can be…
Shannon information theory provides various measures of so-called "syntactic information", which reflect the amount of statistical correlation between systems. In contrast, the concept of "semantic information" refers to those correlations…
In biological systems, information is frequently transferred with Poisson like spike processes (shot noise) modulated in time by information-carrying signals. How then to quantify information transfer for the output for such nonstationary…