Related papers: Linear Information Coupling Problems
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only…
A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…
We give a simple analytic criterion which characterizes linearizable 1-codimensional webs. Then we give an invariant geometrical interpretation of it, in term of projective connection. We explain then how our approach allows to study…
In this paper, we study the information transmission problem under the distributed learning framework, where each worker node is merely permitted to transmit a $m$-dimensional statistic to improve learning results of the target node.…
The problem of detecting network structures plays a central role in distributed computing. One of the fundamental problems studied in this area is to determine whether for a given graph $H$, the input network contains a subgraph isomorphic…
A complex network is a condensed representation of the relational topological framework of a complex system. A main reason for the existence of such networks is the transmission of items through the entities of these complex systems. Here,…
Let $G$ be a complete edge-weighted graph on $n$ vertices. To each subset of vertices of $G$ assign the cost of the minimum spanning tree of the subset as its weight. Suppose that $n$ is a multiple of some fixed positive integer $k$. The…
We consider the two scenarios of communicating a pair $S_{1},S_{2}$ of correlated sources over multiple access (MAC) and interference channels (IC) respectively. We undertake a Shannon theoretic study and focus on achievability, i.e.,…
Clustering large datasets is a fundamental problem with a number of applications in machine learning. Data is often collected on different sites and clustering needs to be performed in a distributed manner with low communication. We would…
This paper studies the problem of embedding very large information networks into low-dimensional vector spaces, which is useful in many tasks such as visualization, node classification, and link prediction. Most existing graph embedding…
Networks of disparate phenomena-- be it the global ecology, human social institutions, within the human brain, or in micro-scale protein interactions-- exhibit broadly consistent architectural features. To explain this, we propose a new…
To what extent can we distinguish one probability distribution from another? Are there quantitative measures of distinguishability? The goal of this tutorial is to approach such questions by introducing the notion of the "distance" between…
Entity-linking is a natural-language-processing task that consists in identifying the entities mentioned in a piece of text, linking each to an appropriate item in some knowledge base; when the knowledge base is Wikipedia, the problem comes…
The presence of symmetries imposes a stringent set of constraints on a system. This constrained structure allows intelligent agents interacting with such a system to drastically improve the efficiency of learning and generalization, through…
A communication network is a graph in which each node has only local information about the graph and nodes communicate by passing messages along its edges. Here, we consider the {\it geometric communication network} where the nodes also…
We study a novel multi-terminal source coding setup motivated by the biclustering problem. Two separate encoders observe two i.i.d. sequences $X^n$ and $Y^n$, respectively. The goal is to find rate-limited encodings $f(x^n)$ and $g(z^n)$…
Several applications in communication, control, and learning require approximating target distributions to within small informational divergence (I-divergence). The additional requirement of invertibility usually leads to using encoders…
We approach the problem of linear network coding for multicast networks from different perspectives. We introduce the notion of the coding points of a network, which are edges of the network where messages combine and coding occurs. We give…
We consider singularly perturbed convection-diffusion equations on one-dimensional networks (metric graphs) as well as the transport problems arising in the vanishing diffusion limit. Suitable coupling condition at inner vertices are…