Related papers: Directed Information, Causal Estimation, and Commu…
In this paper, we study Sibson's $\alpha$-mutual information in the context of the additive Gaussian noise channel. While the classical case $\alpha = 1$ is well understood and admits deep connections to estimation-theoretic quantities,…
A general information transmission model, under independent and identically distributed Gaussian codebook and nearest neighbor decoding rule with processed channel output, is investigated using the performance metric of generalized mutual…
Information transmission over discrete-time channels with memoryless additive noise obeying a Cauchy, rather than Gaussian, distribution, are studied. The channel input satisfies an average power constraint. Upper and lower bounds to such…
A methodology is developed to realized optimal channel input conditional distributions, which maximize the finite-time horizon directed information, for channels with memory and feedback, by information lossless randomized strategies. The…
Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of a linear transform followed by a non-linear (possibly random) function. In general, the linear transforms are defined by matrices and the non-linear…
Shannon-Hartley theorem can accurately calculate the channel capacity when the signal observation time is infinite. However, the calculation of finite-time mutual information, which remains unknown, is essential for guiding the design of…
Two familiar notions of correlation are rediscovered as extreme operating points for simulating a discrete memoryless channel, in which a channel output is generated based only on a description of the channel input. Wyner's "common…
Effecting coordination across remote sites in a distributed system is an essential part of distributed computing, and also an inherent challenge. In 1978, an analysis of communication in asynchronous systems was suggested by Leslie Lamport.…
The most natural way to describe an information-carrying system containing a specific noise is an additive white Gaussian-noise (AWGN) channel. In bosonic quantum systems (especially the Gaussian case), although the classical information…
The question of continuous-versus-discrete information representation in the brain is a fundamental yet unresolved question. Historically, most analyses assume a continuous representation without considering the discrete alternative. Our…
Recent experiments have indicated that many biological systems self-organise near their critical point, which hints at a common design principle. While it has been suggested that information transmission is optimized near the critical…
This paper studies the two-user Gaussian interference channel with half-duplex causal cognition. This channel model consists of two source-destination pairs sharing a common wireless channel. One of the sources, referred to as the…
Information causality was proposed as a physical principle to put upper bound on the accessible information gain in a physical bi-partite communication scheme. Intuitively, the information gain cannot be larger than the amount of classical…
Most communication channels are subjected to noise. One of the goals of Information Theory is to add redundancy in the transmission of information so that the information is transmitted reliably and the amount of information transmitted…
The transmission of classical information over a classical channel gave rise to the classical capacity theorem with the optimal rate in terms of the classical mutual information. Despite classical information being a subset of quantum…
Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about…
For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or…
Computational mechanics quantifies structure in a stochastic process via its causal states, leading to the process's minimal, optimal predictor---the $\epsilon$-machine. We extend computational mechanics to communication channels between…
Quantum capacity gives the fundamental limit of information transmission through a channel. However, evaluating the quantum capacities of a continuous-variable bosonic quantum channel, as well as finding an optimal code to achieve the…
We consider an open quantum system undergoing Markovian dynamics, the latter being modelled by a discrete-time quantum Markov semigroup $(\Phi^n)_{n \in {\mathbb{N}}}$, resulting from the action of sequential uses of a quantum channel…