Related papers: Information without rolling dice
We present a quantum information theory that allows for a consistent description of entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices (rather than probability distributions) for the…
We address the problem of applying the Kolmogorov-Sinai method of entropic analysis, expressed in a generalized non-extensive form, to the dynamics of the logistic map at the chaotic threshold, which is known to be characterized by a power…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
Recent research has demonstrated significant achievable performance gains by exploiting circularity/non-circularity or propeness/improperness of complex-valued signals. In this paper, we investigate the influence of these properties on…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
We show that the entropy of a message can be tested in a device-independent way. Specifically, we consider a prepare-and-measure scenario with classical or quantum communication, and develop two different methods for placing lower bounds on…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
We consider the problem of covert communication with random slot selection over binary-input Discrete Memoryless Channels and Additive White Gaussian Noise channels, in which a transmitter attempts to reliably communicate with a legitimate…
Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients…
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic…
A definition of the nonadditive (nonextensive) conditional entropy indexed by q is presented. Based on the composition law in terms of it, the Shannon-Khinchin axioms are generalized and the uniqueness theorem is established for the Tsallis…
Numerical estimates of the Kolmogorov-Sinai entropy based on a finite amount of data decay towards zero in the relevant limits. Rewriting differences of block entropies as averages over decay rates, and ignoring all parts of the sample…
We calculate Shannon information entropy of trapped interacting bosons in both the position and momentum spaces, $S_r$ and $S_k$ respectively. The total entropy maintains the fuctional form $S=a + b \ln N$ for repulsive bosons. At the…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
Shannon's information theory deliberately excludes message semantics. This paper develops a rigorous framework for semantic communication that integrates formal proof systems with Shannon-theoretic tools. We introduce an axiomatic…
We provide a rigorous first-principle derivation of the non-additive Tsallis' entropy by employing the Chaitin-Kolmogorov algorithmic information theory. By applying non-local restrictive rules on the string formation (grammar), we show…
The deep connection between entropy and information is discussed in terms of both classical and quantum physics. The mechanism of information transfer between systems via entanglement is explored in the context of decoherence theory. The…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…