English
Related papers

Related papers: Optimal Identity Testing with High Probability

200 papers

This paper considers testing a covariance matrix $\Sigma$ in the high dimensional setting where the dimension $p$ can be comparable or much larger than the sample size $n$. The problem of testing the hypothesis $H_0:\Sigma=\Sigma_0$ for a…

Statistics Theory · Mathematics 2013-12-18 T. Tony Cai , Zongming Ma

Distribution testing can be described as follows: $q$ samples are being drawn from some unknown distribution $P$ over a known domain $[n]$. After the sampling process, a decision must be made about whether $P$ holds some property, or is far…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-12-04 Uri Meir

Consider the problem: we are given $n$ boxes, labeled $\{1,2,\ldots, n\}$ by an adversary, each containing a single number chosen from an unknown distribution; these $n$ distributions are not necessarily identical. We are also given an…

Data Structures and Algorithms · Computer Science 2024-05-13 Mohammad Taghi Hajiaghayi , Dariusz R. Kowalski , Piotr Krysta , Jan Olkowski

We study the problem of hypothesis selection under the constraint of local differential privacy. Given a class $\mathcal{F}$ of $k$ distributions and a set of i.i.d. samples from an unknown distribution $h$, the goal of hypothesis selection…

Machine Learning · Statistics 2026-03-05 Alireza F. Pour , Hassan Ashtiani , Shahab Asoodeh

In the hypothesis selection problem, we are given sample and query access to finite set of candidate distributions (hypotheses), $\mathcal{H} = \{H_1, \ldots, H_n\}$, and samples from an unknown distribution $P$, both over a domain…

Data Structures and Algorithms · Computer Science 2025-11-12 Anders Aamand , Maryam Aliakbarpour , Justin Y. Chen , Sandeep Silwal

We study the following distribution clustering problem: Given a hidden partition of $k$ distributions into two groups, such that the distributions within each group are the same, and the two distributions associated with the two clusters…

Data Structures and Algorithms · Computer Science 2025-12-10 Gunjan Kumar , Yash Pote , Jonathan Scarlett

Determining whether an unknown distribution matches a known reference is a cornerstone problem in distributional analysis. While classical results establish a rigorous framework in the case of distributions over finite domains, real-world…

Formal Languages and Automata Theory · Computer Science 2025-08-07 Smayan Agarwal , Shobhit Singh , Aalok Thakkar

We give a nearly-optimal algorithm for testing uniformity of distributions supported on $\{-1,1\}^n$, which makes $\tilde O (\sqrt{n}/\varepsilon^2)$ queries to a subcube conditional sampling oracle (Bhattacharyya and Chakraborty (2018)).…

Data Structures and Algorithms · Computer Science 2021-02-08 Clément L. Canonne , Xi Chen , Gautam Kamath , Amit Levi , Erik Waingarten

We consider the problem of identifying, from statistics, a distribution of discrete random variables $X_1,\ldots,X_n$ that is a mixture of $k$ product distributions. The best previous sample complexity for $n \in O(k)$ was $(1/\zeta)^{O(k^2…

Machine Learning · Computer Science 2023-09-26 Spencer L. Gordon , Erik Jahn , Bijan Mazaheri , Yuval Rabani , Leonard J. Schulman

We present an identity for an unbiased estimate of a general statistical distribution. The identity computes the distribution density from dividing a histogram sum over a local window by a correction factor from a mean-force integral, and…

Computational Physics · Physics 2012-06-04 Cheng Zhang , Jianpeng Ma

We explore the problem of distributed Hypothesis Testing (DHT) against independence, focusing specifically on Binary Symmetric Sources (BSS). Our investigation aims to characterize the optimal quantizer among binary linear codes, with the…

Information Theory · Computer Science 2024-10-23 Fatemeh Khaledian , Reza Asvadi , Elsa Dupraz , Tad Matsumoto

Uncertainty estimation aims to evaluate the confidence of a trained deep neural network. However, existing uncertainty estimation approaches rely on low-dimensional distributional assumptions and thus suffer from the high dimensionality of…

Machine Learning · Computer Science 2023-10-26 Tsai Hor Chan , Kin Wai Lau , Jiajun Shen , Guosheng Yin , Lequan Yu

We investigate the problems of identity and closeness testing over a discrete population from random samples. Our goal is to develop efficient testers while guaranteeing Differential Privacy to the individuals of the population. We describe…

Machine Learning · Computer Science 2017-07-19 Maryam Aliakbarpour , Ilias Diakonikolas , Ronitt Rubinfeld

We revisit the outlier hypothesis testing framework of Li \emph{et al.} (TIT 2014) and derive fundamental limits for the optimal test. In outlier hypothesis testing, one is given multiple observed sequences, where most sequences are…

Statistics Theory · Mathematics 2022-05-17 Lin Zhou , Yun Wei , Alfred Hero

Uniformity testing, or testing whether independent observations are uniformly distributed, is the prototypical question in distribution testing. Over the past years, a line of work has been focusing on uniformity testing under privacy…

Data Structures and Algorithms · Computer Science 2021-10-19 Clément L. Canonne , Hongyi Lyu

As statistical analyses become more central to science, industry and society, there is a growing need to ensure correctness of their results. Approximate correctness can be verified by replicating the entire analysis, but can we verify…

Computational Complexity · Computer Science 2024-09-11 Tal Herman , Guy Rothblum

The objective of goodness-of-fit testing is to assess whether a dataset of observations is likely to have been drawn from a candidate probability distribution. This paper presents a rank-based family of goodness-of-fit tests that is…

Statistics Theory · Mathematics 2019-04-18 Feras A. Saad , Cameron E. Freer , Nathanael L. Ackerman , Vikash K. Mansinghka

In classical statistics and distribution testing, it is often assumed that elements can be sampled from some distribution $P$, and that when an element $x$ is sampled, the probability $P$ of sampling $x$ is also known. Recent work in…

Data Structures and Algorithms · Computer Science 2022-08-03 Talya Eden , Jakob Bæk Tejs Houen , Shyam Narayanan , Will Rosenbaum , Jakub Tětek

We extend the framework of augmented distribution testing (Aliakbarpour, Indyk, Rubinfeld, and Silwal, NeurIPS 2024) to the differentially private setting. This captures scenarios where a data analyst must perform hypothesis testing tasks…

Machine Learning · Computer Science 2025-03-20 Maryam Aliakbarpour , Arnav Burudgunte , Clément Cannone , Ronitt Rubinfeld

Private closeness testing asks to decide whether the underlying probability distributions of two sensitive datasets are identical or differ significantly in statistical distance, while guaranteeing (differential) privacy of the data. As in…

Data Structures and Algorithms · Computer Science 2023-09-14 Clément L. Canonne , Yucheng Sun