Related papers: Directed Information on Abstract Spaces: Propertie…
The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these…
For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or…
How can the information that a set ${X_{1},...,X_{n}}$ of random variables contains about another random variable $S$ be decomposed? To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique…
We consider distributed optimization by a collection of nodes, each having access to its own convex function, whose collective goal is to minimize the sum of the functions. The communications between nodes are described by a time-varying…
Brain networks are adaptively rewired continually, adjusting their topology to bring about functionality and efficiency in sensory, motor and cognitive tasks. In model neural network architectures, adaptive rewiring generates complex,…
In this paper, the distributed strongly convex optimization problem is studied with spatio-temporal compressed communication and equality constraints. For the case where each agent holds an distributed local equality constraint, a…
The representations of conditional entropy and conditional mutual information are significant in explaining the unique effects among variables. While previous studies based on conditional contrastive sampling have effectively removed…
In this paper, we study the asymptotic properties of distributed consensus algorithms over switching directed random networks. More specifically, we focus on consensus algorithms over independent and identically distributed, directed…
It is shown that space-time may possess the differentiability properties of manifolds as well as the ultraviolet finiteness properties of lattices. Namely, if a field's amplitudes are given on any sufficiently dense set of discrete points…
In 2000, Evans et al. [Eva+00] proved the subadditivity of the mutual information in the broadcasting on tree model with binary vertex labels and symmetric channels. They raised the question of whether such subadditivity extends to loopy…
Recent results from statistical physics show that large classes of complex networks, both man-made and of natural origin, are characterized by high clustering properties yet strikingly short path lengths between pairs of nodes. This class…
There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…
Spread of information in crowd is analysed in terms of directed percolation in two-dimensional spatial network. We investigate the case when the information transmitted can be incomplete or damaged. The results indicate that for small or…
The push-sum algorithm allows distributed computing of the average on a directed graph, and is particularly relevant when one is restricted to one-way and/or asynchronous communications. We investigate its behavior in the presence of…
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class…
Interaction information is one of the multivariate generalizations of mutual information, which expresses the amount information shared among a set of variables, beyond the information, which is shared in any proper subset of those…
We restrict the propagation of misinformation in a social-media-like environment while preserving the spread of correct information. We model the environment as a random network of users in which each news item propagates in the network in…
Current discrete randomness and information conservation inequalities are over total recursive functions, i.e. restricted to deterministic processing. This restriction implies that an algorithm can break algorithmic randomness conservation…
This paper studies the distributed optimization problem with possibly nonidentical local constraints, where its global objective function is composed of $N$ convex functions. The aim is to solve the considered optimization problem in a…
Neural networks and other machine learning models compute continuous representations, while humans communicate with discrete symbols. Reconciling these two forms of communication is desirable to generate human-readable interpretations or to…