Related papers: Distribution Estimation under the Infinity Norm

In this paper, we discuss some theoretical results and properties of a discrete version of the Birnbaum-Saunders distribution. We present a proof of the unimodality of this model. Moreover, results on moments, quantile function, reliability…

We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…

We consider an extension of $\epsilon$-entropy to a KL-divergence based complexity measure for randomized density estimation methods. Based on this extension, we develop a general information-theoretical inequality that measures the…

To answer questions of "causes of effects", the probability of necessity is introduced for assessing whether or not an observed outcome was caused by an earlier treatment. However, the statistical inference for probability of necessity is…

Since exact probabilistic inference is intractable in general for large multiply connected belief nets, approximate methods are required. A promising approach is to use heuristic search among hypotheses (instantiations of the network) to…

The estimation of a density profile from experimental data points is a challenging problem, usually tackled by plotting a histogram. Prior assumptions on the nature of the density, from its smoothness to the specification of its form, allow…

Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…

Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…

In this paper, we compute the tightest possible bounds on the probability that the optimal value of a combinatorial optimization problem in maximization form with a random objective exceeds a given number, assuming only knowledge of the…

Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal…

Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…

We study the problem of testing discrete distributions with a focus on the high probability regime. Specifically, given samples from one or more discrete distributions, a property $\mathcal{P}$, and parameters $0< \epsilon, \delta <1$, we…

Given an i.i.d. sample from a distribution $F$ on $\mathbb{R}$ with uniformly continuous density $p_0$, purely data-driven estimators are constructed that efficiently estimate $F$ in sup-norm loss and simultaneously estimate $p_0$ at the…

The classic problems of testing uniformity of and learning a discrete distribution, given access to independent samples from it, are examined under general $\ell_p$ metrics. The intuitions and results often contrast with the classic…

A simple method is shown to provide optimal variational bounds on $f$-divergences with possible constraints on relative information extremums. Known results are refined or proved to be optimal as particular cases.

We establish effective convergence rates in the Doeblin-Lenstra law, describing the limiting distribution of approximation coefficients arising from continued fraction convergents of a typical real number. More generally, we prove…

We propose an iterative estimating equations procedure for analysis of longitudinal data. We show that, under very mild conditions, the probability that the procedure converges at an exponential rate tends to one as the sample size…

We derive an (almost) guaranteed upper bound on the error of deep neural networks under distribution shift using unlabeled test data. Prior methods either give bounds that are vacuous in practice or give estimates that are accurate on…

We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel…

This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an…