English
Related papers

Related papers: Probabilistic Bounds for Data Storage with Feature…

200 papers

We study the density estimation problem defined as follows: given $k$ distributions $p_1, \ldots, p_k$ over a discrete domain $[n]$, as well as a collection of samples chosen from a ``query'' distribution $q$ over $[n]$, output $p_i$ that…

Data Structures and Algorithms · Computer Science 2024-10-31 Anders Aamand , Alexandr Andoni , Justin Y. Chen , Piotr Indyk , Shyam Narayanan , Sandeep Silwal , Haike Xu

We consider the problem of hypothesis testing for discrete distributions. In the standard model, where we have sample access to an underlying distribution $p$, extensive research has established optimal bounds for uniformity testing,…

Machine Learning · Computer Science 2024-12-03 Maryam Aliakbarpour , Piotr Indyk , Ronitt Rubinfeld , Sandeep Silwal

Current discrete randomness and information conservation inequalities are over total recursive functions, i.e. restricted to deterministic processing. This restriction implies that an algorithm can break algorithmic randomness conservation…

Computational Complexity · Computer Science 2013-10-15 Samuel Epstein

This paper studies the \emph{subset sampling} problem. The input is a set $\mathcal{S}$ of $n$ records together with a function $\textbf{p}$ that assigns each record $v\in\mathcal{S}$ a probability $\textbf{p}(v)$. A query returns a random…

Data Structures and Algorithms · Computer Science 2023-07-24 Jinchao Huang , Sibo Wang

The facility location problem is widely used for summarizing large datasets and has additional applications in sensor placement, image retrieval, and clustering. One difficulty of this problem is that submodular optimization algorithms…

Machine Learning · Statistics 2017-03-09 Erik M. Lindgren , Shanshan Wu , Alexandros G. Dimakis

Data compression is an efficient technique to save data storage and transmission costs. However, traditional data compression methods always ignore the impact of user preferences on the statistical distributions of symbols transmitted over…

Information Theory · Computer Science 2019-04-01 Yawei Lu , Wei Chen , H. Vincent Poor

Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent algorithm is widely used and delivers state of the art results for many problems. Nonetheless, Stochastic…

Machine Learning · Computer Science 2018-09-26 Yao Zhang , Andrew M. Saxe , Madhu S. Advani , Alpha A. Lee

We examine the problem of allocating a given total storage budget in a distributed storage system for maximum reliability. A source has a single data object that is to be coded and stored over a set of storage nodes; it is allowed to store…

Information Theory · Computer Science 2016-11-15 Derek Leong , Alexandros G. Dimakis , Tracey Ho

Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate…

Information Theory · Computer Science 2017-09-21 Lin Zhou , Vincent Y. F. Tan , Mehul Motani

We consider the problem of optimally allocating a given total storage budget in a distributed storage system. A source has a data object which it can code and store over a set of storage nodes; it is allowed to store any amount of coded…

Information Theory · Computer Science 2016-11-15 Derek Leong , Alexandros G. Dimakis , Tracey Ho

Let $P = \{p(i)\}$ be a measure of strictly positive probabilities on the set of nonnegative integers. Although the countable number of inputs prevents usage of the Huffman algorithm, there are nontrivial $P$ for which known methods find a…

Information Theory · Computer Science 2016-11-17 Michael B. Baer

Random features provide a practical framework for large-scale kernel approximation and supervised learning. It has been shown that data-dependent sampling of random features using leverage scores can significantly reduce the number of…

Machine Learning · Computer Science 2019-03-21 Shahin Shahrampour , Soheil Kolouri

In this paper, we consider the problem of column subset selection. We present a novel analysis of the spectral norm reconstruction for a simple randomized algorithm and establish a new bound that depends explicitly on the sampling…

Numerical Analysis · Mathematics 2015-05-05 Tianbao Yang , Lijun Zhang , Rong Jin , Shenghuo Zhu

There exist many high-dimensional data in real-world applications such as biology, computer vision, and social networks. Feature selection approaches are devised to confront with high-dimensional data challenges with the aim of efficient…

Machine Learning · Computer Science 2021-06-22 Mohsen Ghassemi Parsa , Hadi Zare , Mehdi Ghatee

We investigate lossy compression (source coding) of data in the form of permutations. This problem has direct applications in the storage of ordinal data or rankings, and in the analysis of sorting algorithms. We analyze the rate-distortion…

Information Theory · Computer Science 2016-11-18 Da Wang , Arya Mazumdar , Gregory Wornell

A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…

Information Theory · Computer Science 2026-04-16 Terence Viaud , Ioannis Kontoyiannis

We study the inherent space requirements of shared storage algorithms in asynchronous fault-prone systems. Previous works use codes to achieve a better storage cost than the well-known replication approach. However, a closer look reveals…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-07-21 Alexander Spiegelman , Yuval Cassuto , Gregory Chockler , Idit Keidar

Hyperspectral data consists of large number of features which require sophisticated analysis to be extracted. A popular approach to reduce computational cost, facilitate information representation and accelerate knowledge discovery is to…

Machine Learning · Computer Science 2015-09-29 Phool Preet , Sanjit Singh Batra , Jayadeva

Logconcave functions represent the current frontier of efficient algorithms for sampling, optimization and integration in R^n. Efficient sampling algorithms to sample according to a probability density (to which the other two problems can…

Data Structures and Algorithms · Computer Science 2009-06-16 Karthekeyan Chandrasekaran , Amit Deshpande , Santosh Vempala

We suggest a robust nearest-neighbor approach to classifying high-dimensional data. The method enhances sensitivity by employing a threshold and truncates to a sequence of zeros and ones in order to reduce the deleterious impact of…

Statistics Theory · Mathematics 2009-09-02 Yao-ban Chan , Peter Hall