Related papers: Which entropy for general physical theories?
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
A direct connection of information entropy $S$ and kinetic energy $T$ is obtained for nuclei and atomic clusters, which establishes $T$ as a measure of the information in a distribution. It is conjectured that this is a universal property…
A system of interacting qubits can be viewed as a non-i.i.d quantum information source. A possible model of such a source is provided by a quantum spin system, in which spin-1/2 particles located at sites of a lattice interact with each…
Due to the absence of an external, classical time variable, the probabilistic predictions of covariant quantum theory are ambiguous when multiple measurements are considered. Here, we introduce an information theoretic framework to the…
Researchers have proposed formal definitions of quantitative information flow based on information theoretic notions such as the Shannon entropy, the min entropy, the guessing entropy, and channel capacity. This paper investigates the…
Complex systems that are characterized by strong correlations and fat-tailed distribution functions have been argued to be incompatible within the framework of Boltzmann-Gibbs entropy. As an alternative, so-called generalized entropies were…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
In classical information theory, entropy rate and Kolmogorov complexity per symbol are related by a theorem of Brudno. In this paper, we prove a quantum version of this theorem, connecting the von Neumann entropy rate and two notions of…
Thermodynamic entropy is not an entirely satisfactory measure of information of a quantum state. This entropy for an unknown pure state is zero, although repeated measurements on copies of such a pure state do communicate information. In…
Quantum information theory gives rise to a straightforward definition of the interaction of electrons $I_{p,q}$ in two orbitals $p$, $q$ for a given many-body wave function. A convenient way to calculate the von Neumann entropies needed is…
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two…
We show that the main difference between classical and quantum systems can be understood in terms of information entropy. Classical systems can be considered the ones where the internal dynamics can be known with arbitrary precision while…
According to quantum mechanics, the informational content of isolated systems does not change in time. However, subadditivity of entropy seems to describe an excess of information when we look at single parts of a composite systems and…
We consider entropy in Generalized Non-Signalling Theory (also known as box world) where the most common definition of entropy is the measurement entropy. In this setting, we completely characterize the set of allowed entropies for a…
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
Entropy and other fundamental quantities of information theory are customarily expressed and manipulated as functions of probabilities. Here we study the entropy H and subentropy Q as functions of the elementary symmetric polynomials in the…
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…