English
Related papers

Related papers: Detecting Correlations with Little Memory and Comm…

200 papers

The problem of statistical inference in its various forms has been the subject of decades-long extensive research. Most of the effort has been focused on characterizing the behavior as a function of the number of available samples, with far…

Machine Learning · Computer Science 2024-11-12 Tomer Berg , Or Ordentlich , Ofer Shayevitz

We prove an optimal $\Omega(n)$ lower bound on the randomized communication complexity of the much-studied Gap-Hamming-Distance problem. As a consequence, we obtain essentially optimal multi-pass space lower bounds in the data stream model…

Computational Complexity · Computer Science 2012-07-02 Amit Chakrabarti , Oded Regev

We study how the amount of correlation between observations collected by distinct sensors/learners affects data collection and collaboration strategies by analyzing Fisher information and the Cramer-Rao bound. In particular, we consider a…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-06-02 Yu-Zhen Janice Chen , Daniel S. Menasche , Don Towsley

Neural networks encode information through their collective spiking activity in response to external stimuli. This population response is noisy and strongly correlated, with complex interplay between correlations induced by the stimulus,…

Neurons and Cognition · Quantitative Biology 2022-11-28 Gabriel Mahuas , Olivier Marre , Thierry Mora , Ulisse Ferrari

We study the problem of communicating a distributed correlated memoryless source over a memoryless network, from source nodes to destination nodes, under quadratic distortion constraints. We establish the following two complementary…

Information Theory · Computer Science 2013-04-09 Himanshu Asnani , Ilan Shomorony , A. Salman Avestimehr , Tsachy Weissman

To construct models of large, multivariate complex systems, such as those in biology, one needs to constrain which variables are allowed to interact. This can be viewed as detecting "local" structures among the variables. In the context of…

Data Analysis, Statistics and Probability · Physics 2023-10-19 Mahajabin Rahman , Ilya Nemenman

We consider a private discrete distribution estimation problem with one-bit communication constraint. The privacy constraints are imposed with respect to the local differential privacy and the maximal leakage. The estimation error is…

Information Theory · Computer Science 2023-10-18 Seung-Hyun Nam , Vincent Y. F. Tan , Si-Hyeon Lee

We formulate the problem of performing optimal data compression under the constraints that compressed data can be used for accurate classification in machine learning. We show that this translates to a problem of minimizing the mutual…

Signal Processing · Electrical Eng. & Systems 2022-11-04 Jingchao Gao , Ao Tang , Weiyu Xu

Correlation analysis is a fundamental problem in statistics. In this paper, we consider the correlation detection problem between a pair of Erdos-Renyi graphs. Specifically, the problem is formulated as a hypothesis testing problem: under…

Statistics Theory · Mathematics 2026-01-21 Dong Huang , Pengkun Yang

In modern data center networks, thousands of hosts contend for shared link capacity; the scale of these systems makes centralized scheduling impractical. This article models such scheduling as a bipartite matching problem under…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-04-14 Moonmoon Mohanty , Gautham Bolar , Preetam Patil , Ayalvadi Ganesh , Jean-Francois Chamberland , Parimal Parag

We consider algorithmic problems in the setting in which the input data has been partitioned arbitrarily on many servers. The goal is to compute a function of all the data, and the bottleneck is the communication used by the algorithm. We…

Data Structures and Algorithms · Computer Science 2014-07-01 Ravindran Kannan , Santosh Vempala , David Woodruff

Quantum communication leads to strong correlations, that can outperform classical ones. Complementary to previous works in this area, we investigate correlations in prepare-and-measure scenarios assuming a bound on the information content…

This paper is concerned with the analysis of correlation between two high-dimensional data sets when there are only few correlated signal components but the number of samples is very small, possibly much smaller than the dimensions of the…

Information Theory · Computer Science 2016-04-08 Yang Song , Peter J. Schreier , David Ramirez , Tanuj Hasija

In this work, we show, for the well-studied problem of learning parity under noise, where a learner tries to learn $x=(x_1,\ldots,x_n) \in \{0,1\}^n$ from a stream of random linear equations over $\mathrm{F}_2$ that are correct with…

Machine Learning · Computer Science 2021-07-07 Sumegha Garg , Pravesh K. Kothari , Pengda Liu , Ran Raz

This thesis is concerned with the design of distributed algorithms for solving optimization problems. We consider networks where each node has exclusive access to a cost function, and design algorithms that make all nodes cooperate to find…

Optimization and Control · Mathematics 2013-12-03 João F. C. Mota

We investigate the minimal number of failures that can partition a system where processes communicate both through shared memory and by message passing. We prove that this number precisely captures the resilience that can be achieved by…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-12-22 Hagit Attiya , Sweta Kumari , Noa Schiller

Associative memories are structures that store data in such a way that it can later be retrieved given only a part of its content -- a sort-of error/erasure-resilience property. They are used in applications ranging from caches and memory…

Information Theory · Computer Science 2013-04-23 Vincent Gripon , Michael Rabbat

The problem of statistical learning is to construct an accurate predictor of a random variable as a function of a correlated random variable on the basis of an i.i.d. training sample from their joint distribution. Allowable predictors are…

Information Theory · Computer Science 2009-04-30 Maxim Raginsky

We study hypothesis testing under communication constraints, where each sample is quantized before being revealed to a statistician. Without communication constraints, it is well known that the sample complexity of simple binary hypothesis…

Statistics Theory · Mathematics 2023-12-19 Ankit Pensia , Varun Jog , Po-Ling Loh

We consider the communication complexity of a number of distributed optimization problems. We start with the problem of solving a linear system. Suppose there is a coordinator together with $s$ servers $P_1, \ldots, P_s$, the $i$-th of…

Data Structures and Algorithms · Computer Science 2019-11-01 Santosh S. Vempala , Ruosong Wang , David P. Woodruff