Related papers: Analysis of an information-theoretic model for com…
The \emph{beep model} is a very weak communications model in which devices in a network can communicate only via beeps and silence. As a result of its weak assumptions, it has broad applicability to many different implementations of…
The question of how much communication is required between collaborating parties to compute a function of their data is of fundamental importance in the fields of theoretical computer science and information theory. In this work, the focus…
In this paper, we consider the problem of minimizing the total transmission cost for exchanging channel state information. We proposed a network coded cooperative data exchange scheme, such that the total transmission cost is minimized…
Ferrer-i-Cancho (2015) presents a mathematical model of both the synchronic and diachronic nature of word order based on the assumption that memory costs are a never decreasing function of distance and a few very general linguistic…
Thermodynamic cost of communication is a major factor in the thermodynamic cost of real-world computers, both biological and digital. Despite its importance, the fundamental principles underlying this cost remain poorly understood. This…
Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. However, compression can introduce information distortion,…
The communication cost of algorithms (also known as I/O-complexity) is shown to be closely related to the expansion properties of the corresponding computation graphs. We demonstrate this on Strassen's and other fast matrix multiplication…
The task of text segmentation may be undertaken at many levels in text analysis---paragraphs, sentences, words, or even letters. Here, we focus on a relatively fine scale of segmentation, hypothesizing it to be in accord with a stochastic…
n-tuple power law widely exists in language, computer program code, DNA and music. After a vast amount of Zipf analyses of n-tuple power law from empirical data, we propose a model to explain the n-tuple power law feature existed in these…
Upper bounds on the communication complexity of finding the nearest lattice point in a given lattice $\Lambda \subset \mathbb{R}^2$ was considered in earlier works~\cite{VB:2017}, for a two party, interactive communication model. Here we…
We develop a novel framework for costly information acquisition in which a decision-maker learns about an unobserved state by choosing a signal distribution, with the cost of information determined by the distribution of noise in the…
Present human languages display slightly asymmetric log-normal (Gauss) distribution for size [1-3], whereas present cities follow power law (Pareto-Zipf law)[4]. Our model considers the competition between languages and that between cities…
I study a model of costly Bayesian persuasion by a privately and partially informed sender who conducts a public experiment. The cost of running an experiment is the expected reduction of a weighted log-likelihood ratio function of the…
The nearest lattice point problem in $\mathbb{R}^n$ is formulated in a distributed network with $n$ nodes. The objective is to minimize the probability that an incorrect lattice point is found, subject to a constraint on inter-node…
Zipf's law predicts a power-law relationship between word rank and frequency in language communication systems and has been widely reported in a variety of natural language processing applications. However, the emergence of natural language…
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive empirical study of the statistics of model updates in FL,…
We study the communication complexity of symmetric XOR functions, namely functions $f: \{0,1\}^n \times \{0,1\}^n \rightarrow \{0,1\}$ that can be formulated as $f(x,y)=D(|x\oplus y|)$ for some predicate $D: \{0,1,...,n\} \rightarrow…
We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…
The problem of compression in standard information theory consists of assigning codes as short as possible to numbers. Here we consider the problem of optimal coding -- under an arbitrary coding scheme -- and show that it predicts Zipf's…
We study the bounds of mediated communication in sender-receiver games in which the sender's payoff is state-independent. We show that the feasible distributions over the receiver's beliefs under mediation are those that induce zero…