Related papers: Directed Information on Abstract spaces: Propertie…
Directed information or its variants are utilized extensively in the characterization of the capacity of channels with memory and feedback, nonanticipative lossy data compression, and their generalizations to networks. In this paper, we…
A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the…
The main theme of this thesis is the development of computational methods for classes of infinite-dimensional optimization problems arising in optimal control and information theory. The first part of the thesis is concerned with the…
In many areas of applied mathematics, engineering, and social and natural sciences, decentralization of information is a key aspect determining how to approach a problem. In this review article, we study information structures in a…
The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures,…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
We describe a convex programming approach to the calculation of lower bounds on the minimum cost of constrained decentralized control problems with nonclassical information structures. The class of problems we consider entail the…
Directed information (DI) is an information measure that attempts to capture directionality in the flow of information from one random process to another. It is closely related to other causal influence measures, such as transfer entropy,…
Powerspaces of directed spaces play an important role in modeling the semantics of nondeterministic functional programming languages. The notions of upper,lower and convex powerspace of a directed space are defined by the way of free…
In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information…
Low-dimensional representations of underdamped systems often provide insightful grasps and analytical tractability. Here, we build such representations via information projections, obtaining an optimal model that captures the most…
The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial…
In information theory, some optimization problems result in convex optimization problems on strictly convex functionals of probability densities. In this note, we study these problems and show conditions of minimizers and the uniqueness of…
Spread of information in crowd is analysed in terms of directed percolation in two-dimensional spatial network. We investigate the case when the information transmitted can be incomplete or damaged. The results indicate that for small or…
We study the problem of computing the capacity of a discrete memoryless channel under uncertainty affecting the channel law matrix, and possibly with a constraint on the average cost of the input distribution. The problem has been…
Directed information (DI) is a fundamental measure for the study and analysis of sequential stochastic models. In particular, when optimized over input distributions it characterizes the capacity of general communication channels. However,…
Recently, there has been an increasing interest in designing distributed convex optimization algorithms under the setting where the data matrix is partitioned on features. Algorithms under this setting sometimes have many advantages over…
For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or…
We study the decentralized consensus and stochastic optimization problems with compressed communications over static directed graphs. We propose an iterative gradient-based algorithm that compresses messages according to a desired…
Distributed abstract programs are a novel class of distributed optimization problems where (i) the number of variables is much smaller than the number of constraints and (ii) each constraint is associated to a network node. Abstract…