相关论文: Directed Data-Processing Inequalities for Systems …
We consider a collection of distributed units that interact with one another through the sending of messages. Each message carries a positive ($+1$) or negative ($-1$) tag and causes the receiving unit to send out messages as a function of…
Information Causality is a physical principle which states that the amount of randomly accessible data over a classical communication channel cannot exceed its capacity, even if the sender and the receiver have access to a source of…
Cells can be considered as systems that utilize changes in thermodynamic entropy as information. Therefore, they serve as useful models for investigating the relationships between entropy production and information transmission, i.e.,…
Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a…
We introduce the (private) entropy of a directed graph (in a new network coding sense) as well as a number of related concepts. We show that the entropy of a directed graph is identical to its guessing number and can be bounded from below…
In this paper, a new problem of transmitting information over the adversarial insertion-deletion channel with feedback is introduced. Suppose that the encoder transmits $n$ binary symbols one-by-one over a channel, in which some symbols can…
The processing of quantum information is limited by fundamental physical constraints on how information can be encoded, transmitted, and extracted. In particular, the non-orthogonality of quantum states limits their distinguishability, and…
Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the…
Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical…
We consider a finite-state memoryless channel with i.i.d. channel state and the input Markov process supported on a mixing finite-type constraint. We discuss the asymptotic behavior of entropy rate of the output hidden Markov chain and…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
Directed information or its variants are utilized extensively in the characterization of the capacity of channels with memory and feedback, nonanticipative lossy data compression, and their generalizations to networks. In this paper, we…
Quantum information theory studies the fundamental limits that physical laws impose on information processing tasks such as data compression and data transmission on noisy channels. This thesis presents general techniques that allow one to…
In this thesis we consider the problem of information hiding in the scenarios of interactive systems, statistical disclosure control, and refinement of specifications. We apply quantitative approaches to information flow in the first two…
We study memoryless interference channels with gradual data arrival in the absence of feedback. The information bits arrive at the transmitters according to independent and asynchronous~(Tx-Tx asynchrony) Bernoulli processes with average…
Channel capacities quantify the optimal rates of sending information reliably over noisy channels. Usually, the study of capacities assumes that the circuits which sender and receiver use for encoding and decoding consist of perfectly…
The traditional information theoretic approach to studying feedback is to consider ideal instantaneous high-rate feedback of the channel outputs to the encoder. This was acceptable in classical work because the results were negative:…
The paper is focused on the tradeoff between performance and decoding complexity per iteration for LDPC codes in terms of their gap (in rate) to capacity. The study of this tradeoff is done via information-theoretic bounds which also enable…
As the mobile application landscape expands, wireless networks are tasked with supporting various connection profiles, including real-time communications and delay-sensitive traffic. Among many ensuing engineering challenges is the need to…
The problem of lossless data compression with side information available to both the encoder and the decoder is considered. The finite-blocklength fundamental limits of the best achievable performance are defined, in two different versions…