Related papers: The Birthday Problem and Zero-Error List Codes
Error probabilities of random codes for memoryless channels are considered in this paper. In the area of communication systems, admissible error probability is very small and it is sometimes more important to discuss the relative gap…
Information bottleneck (IB) and privacy funnel (PF) are two closely related optimization problems which have found applications in machine learning, design of privacy algorithms, capacity problems (e.g., Mrs. Gerber's Lemma), strong data…
We consider a variant of the channel simulation problem with a single input and multiple outputs, where Alice observes a probability distribution $P$ from a set of prescribed probability distributions $\mathbb{\mathcal{P}}$, and sends a…
Random number generators are widely used in practical algorithms. Examples include simulation, number theory (primality testing and integer factorization), fault tolerance, routing, cryptography, optimization by simulated annealing, and…
The problem of lossless data compression with side information available to both the encoder and the decoder is considered. The finite-blocklength fundamental limits of the best achievable performance are defined, in two different versions…
The zero-error channel capacity is the maximum asymptotic rate that can be reached with error probability exactly zero, instead of a vanishing error probability. The nature of this problem, essentially combinatorial rather than…
The information decomposition problem requires an additive decomposition of the mutual information between the input and target variables into nonnegative terms. The recently introduced solution to this problem, Information Attribution,…
Although information theoretic characterizations of human communication have become increasingly popular in linguistics, to date they have largely involved grafting probabilistic constructs onto older ideas about grammar. Similarities…
Exponentiation makes the difference between the bit-size of this line and the number (<< 2^{300}) of particles in the known Universe. The expulsion of exponential time algorithms from Computer Theory in the 60's broke its umbilical cord…
We consider visible compression for discrete memoryless sources of mixed quantum states when only classical information can be sent from Alice to Bob. We assume that Bob knows the source statistics, and that Alice and Bob have identical…
This note is intended to foster a discussion about the extent to which typical problems arising in quantum information theory are algorithmically decidable (in principle rather than in practice). Various problems in the context of…
Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain…
The paper is focused on the tradeoff between performance and decoding complexity per iteration for LDPC codes in terms of their gap (in rate) to capacity. The study of this tradeoff is done via information-theoretic bounds which also enable…
We introduce a universal quantization scheme based on random coding, and we analyze its performance. This scheme consists of a source-independent random codebook (typically_mismatched_ to the source distribution), followed by optimal…
The cocktail party problem refers to the famous selective attention problem of how to find out the signal of each individual sources from signals of a number of detectors. In the classical cocktail party problem, the signal of each source…
Shannon's seminal 1948 work gave rise to two distinct areas of research: information theory and mathematical coding theory. While information theory has had a strong influence on theoretical neuroscience, ideas from mathematical coding…
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…
In theory, the probabilistic linkage method provides two distinct advantages over non-probabilistic methods, including minimal rates of linkage error and accurate measures of these rates for data users. However, implementations can fall…
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules…
Notwithstanding various attempts to construct a Partial Information Decomposition (PID) for multiple variables by defining synergistic, redundant, and unique information, there is no consensus on how one ought to precisely define either of…