Related papers: Foundations of information theory for coding theor…
The more than thirty years old issue of the (classical) information capacity of quantum communication channels was dramatically clarified during the last years, when a number of direct quantum coding theorems was discovered. The present…
In the first paper of this two part communication, we solved in a unified framework a variety of two terminal source coding problems with noncooperative encoders, thereby consolidating works of Shannon, Slepian-Wolf, Wyner,…
Researchers have proposed formal definitions of quantitative information flow based on information theoretic notions such as the Shannon entropy, the min entropy, the guessing entropy, belief, and channel capacity. This paper investigates…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
Shannon introduced the classic model of a cryptosystem in 1949, where Eve has access to an identical copy of the cyphertext that Alice sends to Bob. Shannon defined perfect secrecy to be the case when the mutual information between the…
We extend present Shannon's static statistical information theory to dynamic processes and establish a dynamic statistical information theory. We derive the nonlinear evolution equations of dynamic information density and dynamic…
During the last two decades, concentration inequalities have been the subject of exciting developments in various areas, including convex geometry, functional analysis, statistical physics, high-dimensional statistics, pure and applied…
The weak converse coding theorems have been proved for the quantum source and channel. The results give the lower bound for capacity of source and the upper bound for capacity of channel. The monotonicity of mutual quantum information have…
Claude Shannons information theory (1949) has had a revolutionary impact on communication science. A crucial property of his framework is that it decouples the meaning of a message from the mechanistic details from the actual communication…
Traditional communication theory focuses on minimizing transmit power. However, communication links are increasingly operating at shorter ranges where transmit power can be significantly smaller than the power consumed in decoding. This…
Coding theorems in quantum Shannon theory express the ultimate rates at which a sender can transmit information over a noisy quantum channel. More often than not, the known formulas expressing these transmission rates are intractable,…
Semantic communication has emerged as a promising paradigm to address the challenges of next-generation communication networks. While some progress has been made in its conceptualization, fundamental questions remain unresolved. In this…
Generative models that maximize model likelihood have gained traction in many practical settings. Among them, perturbation based approaches underpin many strong likelihood estimation models, yet they often face slow convergence and limited…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Information theory tells us that if the rate of sending information across a noisy channel were above the capacity of that channel, then the transmission would necessarily be unreliable. For classical information sent over classical or…
Identification of latent binary sequences from a pool of noisy observations has a wide range of applications in both statistical learning and population genetics. Each observed sequence is the result of passing one of the latent…
We define the concept of dependence among multiple variables using maximum entropy techniques and introduce a graphical notation to denote the dependencies. Direct inference of information theoretic quantities from data uncovers…
We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative…
Noisy channels are a foundational resource for constructing cryptographic primitives such as string commitment and oblivious transfer. The noisy channel model has been extended to unfair noisy channels, where adversaries can influence the…
Quantum error-correcting codes are analyzed from an information-theoretic perspective centered on quantum conditional and mutual entropies. This approach parallels the description of classical error correction in Shannon theory, while…