Related papers: Extension of the Blahut-Arimoto algorithm for maxi…
An approach is established for maximizing the Lower bound on the Mismatch capacity (hereafter abbreviated as LM rate), a key performance bound in mismatched decoding, by optimizing the channel input probability distribution. Under a fixed…
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or…
Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer models of the likelihood-to-evidence…
The Arimoto--Blahut algorithm for computing the capacity of a discrete memoryless channel is revisited. A so-called ``squeezing'' strategy is used to design algorithms that preserve its simplicity and monotonic convergence properties, but…
In this paper, we consider a multiple-input multiple-output (MIMO) radar system for localizing a target based on its reflected echo signals. Specifically, we aim to estimate the random and unknown angle information of the target, by…
An information collection problem in a wireless network with random events is considered. Wireless devices report on each event using one of multiple reporting formats. Each format has a different quality and uses different data lengths.…
Energy efficient information transmission may be relevant to biological sensory signal processing as well as to low power electronic devices. We explore its consequences in two different regimes. In an ``immediate'' regime, we argue that…
The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The…
This paper proposes an adaptive multi-mode transmission strategy to improve the spectral efficiency achieved in the multiple-input multiple-output (MIMO) broadcast channel with delayed and quantized channel state information. The adaptive…
The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions…
Bounded agents are limited by intrinsic constraints on their ability to process information that is available in their sensors and memory and choose actions and memory updates. In this dissertation, we model these constraints as…
Reliable communication over a discrete memoryless channel with the help of a relay has aroused interest due to its widespread applications in practical scenarios. By considering the system with a mismatched decoder, previous works have…
The investigation of input-output systems often requires a sophisticated choice of test inputs to make best use of limited experimental time. Here we present an iterative algorithm that continuously adjusts an ensemble of test inputs…
One of the fundamental problems of information theory, since its foundation by Shannon in 1948, has been the computation of the capacity of a discrete memoryless channel, a quantity expressing the maximum rate at which information can…
We introduce a general framework for treating channels with memory and feedback. First, we generalize Massey's concept of directed information and use it to characterize the feedback capacity of general channels. Second, we present coding…
We formulate and analyze the compound information bottleneck programming. In this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow \mathsf{Z} $ is assumed with fixed marginal distributions $\mathsf{P}_{\mathsf{X}}$…
Directed information (DI) is an information measure that attempts to capture directionality in the flow of information from one random process to another. It is closely related to other causal influence measures, such as transfer entropy,…
This study presents alternating optimization (AO) algorithms for computing $\alpha$-mutual information ($\alpha$-MI) and $\alpha$-capacity based on variational characterizations of $\alpha$-MI using a reverse channel. Specifically, we…
A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the…
Modern applied optimization problems become more and more complex every day. Due to this fact, distributed algorithms that can speed up the process of solving an optimization problem through parallelization are of great importance. The main…