Related papers: A simpler derivation of the coding theorem
Based on various strategies and a new general doubling operator, we obtain several simple proofs of the celebrated Sharkovsky's cycle coexistence theorem. A simple non-directed graph proof which is especially suitable for a calculus course…
We give a simple proof of Strassen's theorem on stochastic dominance using linear programming duality, without requiring measure-theoretic arguments. The result extends to generalized inequalities using conic optimization duality and…
Inferring the causal structure that links n observables is usually based upon detecting statistical dependences and choosing simple graphs that make the joint measure Markovian. Here we argue why causal inference is also possible when only…
In this paper, we take a unified approach for network information theory and prove a coding theorem, which can recover most of the achievability results in network information theory that are based on random coding. The final single-letter…
This paper begins with a discussion of integration over probability types (p-types). After doing that, the paper re-visits 3 mainstay problems of classical (non-quantum) Shannon Information Theory (SIT): source coding without distortion,…
Based on various strategies, we obtain several simple proofs of the celebrated Sharkovsky cycle coexistence theorem.
The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the…
We present a quantum information theory that allows for a consistent description of entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices (rather than probability distributions) for the…
A class of information inequalities, called Shannon-type inequalities (STIs), can be proven via a computer software called ITIP. In previous work, we have shown how this technique can be utilized to Fourier-Motzkin elimination algorithm for…
The unit-derived method in coding theory is shown to be a unique optimal scheme for constructing and analysing codes. In many cases efficient and practical decoding methods are produced. Codes with efficient decoding algorithms at maximal…
We study the problem of efficient compression of a stochastic source of probability distributions. It can be viewed as a generalization of Shannon's source coding problem. It has relation to the theory of common randomness, as well as to…
We discuss linear programming techniques that help to deduce corollaries of non classic inequalities for Shannon's entropy. We focus on direct applications of the copy lemma. These applications involve implicitly some (known or unknown)…
We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the…
We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories:…
In this note, we present a simple directed graph proof of Sharkovsky's theorem.
The Chernoff bound is one of the most widely used tools in theoretical computer science. It's rare to find a randomized algorithm that doesn't employ a Chernoff bound in its analysis. The standard proofs of Chernoff bounds are beautiful but…
In this short paper, we shall provide a dynamical systems' proof of the famous Kraft-McMillan inequality and its converse. Kraft-McMillan inequality is a basic result in information theory which gives a necessary and sufficient condition…
The Shannon-Khinchin axioms are generalized to nonextensive systems and the uniqueness theorem for the nonextensive entropy is proved rigorously. In the present axioms, Shannon additivity is used as additivity in contrast to…
We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however…
The likelihood encoder with a random codebook is demonstrated as an effective tool for source coding. Coupled with a soft covering lemma (associated with channel resolvability), likelihood encoders yield simple achievability proofs for…