Related papers: Tight Memory-Independent Parallel Matrix Multiplic…
In recent years, as data and problem sizes have increased, distributed learning has become an essential tool for training high-performance models. However, the communication bottleneck, especially for high-dimensional data, is a challenge.…
As artificial intelligence systems spread to more diverse and larger tasks in many domains, the machine learning algorithms, and in particular the deep learning models and the databases required to train them are getting bigger themselves.…
Cellular automata are one-dimensional arrays of interconnected interacting finite automata. We investigate one of the weakest classes, the real-time one-way cellular automata, and impose an additional restriction on their inter-cell…
Lattice reduction-aided decoding features reduced decoding complexity and near-optimum performance in multi-input multi-output communications. In this paper, a quantitative analysis of lattice reduction-aided decoding is presented. To this…
We investigate query-to-communication lifting theorems for models related to the quantum adversary bounds. Our results are as follows: 1. We show that the classical adversary bound lifts to a lower bound on randomized communication…
We show tight lower bounds for the entire trade-off between space and query time for the Approximate Near Neighbor search problem. Our lower bounds hold in a restricted model of computation, which captures all hashing-based approaches. In…
In this paper, we revisit the communication vs. distributed computing trade-off, studied within the framework of MapReduce in [1]. An implicit assumption in the aforementioned work is that each server performs all possible computations on…
In the era of big data, it is necessary to split extremely large data sets across multiple computing nodes and construct estimators using the distributed data. When designing distributed estimators, it is desirable to minimize the amount of…
Communication compression is an essential strategy for alleviating communication overhead by reducing the volume of information exchanged between computing nodes in large-scale distributed stochastic optimization. Although numerous…
We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where $M$ machines work in parallel over the course of $R$ rounds of communication to optimize…
This paper considers the problem of covert communication with mismatched decoding, in which a sender wishes to reliably communicate with a receiver whose decoder is fixed and possibly sub-optimal, and simultaneously to ensure that the…
Meshless methods approximate operators in a specific node as a weighted sum of values in its neighbours. Higher order approximations of derivatives provide more accurate solutions with better convergence characteristics, but they come at…
We consider differentially private approximate singular vector computation. Known worst-case lower bounds show that the error of any differentially private algorithm must scale polynomially with the dimension of the singular vector. We are…
The ability of machine learning (ML) algorithms to generalize well to unseen data has been studied through the lens of information theory, by bounding the generalization error with the input-output mutual information (MI), i.e., the MI…
This paper considers the problem of decentralized submodular maximization subject to partition matroid constraint using a sequential greedy algorithm with probabilistic inter-agent message-passing. We propose a communication-aware framework…
This document collects the lecture notes from my course "Communication Complexity (for Algorithm Designers),'' taught at Stanford in the winter quarter of 2015. The two primary goals of the course are: 1. Learn several canonical problems…
We develop model free PAC performance guarantees for multiple concurrent MDPs, extending recent works where a single learner interacts with multiple non-interacting agents in a noise free environment. Our framework allows noisy and resource…
Recent advances in model predictive control (MPC) leverage local communication constraints to produce localized MPC algorithms whose complexities scale independently of total network size. However, no characterization is available regarding…
We describe several features of parallel or distributed asynchronous iterative algorithms such as unbounded delays, possible out of order messages or flexible communication. We concentrate on the concept of macroiteration sequence which was…
Resource-constrained systems are prevalent in communications. Such a system is composed of many components but only some of them can be allocated with resources such as time slots. According to the amount of information about the system,…