Related papers: Speeding up lower bound estimation in powerlaw dis…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
We consider the problem of evaluating representations of data for use in solving a downstream task. We propose to measure the quality of a representation by the complexity of learning a predictor on top of the representation that achieves…
We show how recent results by Bening and Korolev in the context of estimation, when linked with a classical result of Fisher concerning the negative binomial distribution, can be used to explain the ubiquity of power law probability…
In this paper, we propose a new class of parameter estimation laws for adaptive systems, called \emph{normalized parameter estimation laws}. A key feature of these estimation laws is that they accelerate the convergence of the system state,…
This paper proposes an easy-to-compute upper bound for the overlap index between two probability distributions without requiring any knowledge of the distribution models. The computation of our bound is time-efficient and memory-efficient…
It has been repeatedly stated that maximum likelihood (ML) estimates of exponents of power-law distributions can only be reliably obtained for exponents smaller than minus one. The main argument that power laws are otherwise not…
Application of the minimum distance method to the linear regression model for estimating regression parameters is a difficult and time-consuming process due to the complexity of its distance function, and hence, it is computationally…
We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…
In this paper, we consider partial sums of martingale differences weighted by random variables drawn uniformly on the sphere, and globally independent of the martingale differences. Combining Lindeberg's method and a series of arguments due…
We propose a communication-efficient distributed estimation method for sparse linear discriminant analysis (LDA) in the high dimensional regime. Our method distributes the data of size $N$ into $m$ machines, and estimates a local sparse LDA…
This paper considers estimation and inference in semiparametric econometric models. Standard procedures estimate the model based on an independence restriction that induces a minimum distance between a joint cumulative distribution function…
Linear regression with normally distributed errors - including particular cases such as ANOVA, Student's t-test or location-scale inference - is a widely used statistical procedure. In this case the ordinary least squares estimator…
We study the density estimation problem defined as follows: given $k$ distributions $p_1, \ldots, p_k$ over a discrete domain $[n]$, as well as a collection of samples chosen from a ``query'' distribution $q$ over $[n]$, output $p_i$ that…
In this paper, a distributed stochastic approximation algorithm is studied. Applications of such algorithms include decentralized estimation, optimization, control or computing. The algorithm consists in two steps: a local step, where each…
We propose a new approach for estimating the parameters of a probability distribution. It consists on combining two new methods of estimation. The first is based on the definition of a new distance measuring the difference between…
The growing amount of high dimensional data in different machine learning applications requires more efficient and scalable optimization algorithms. In this work, we consider combining two techniques, parallelism and Nesterov's…
In this paper, we propose a distributed Newton method for consensus optimization. Our approach outperforms state-of-the-art methods, including ADMM. The key idea is to exploit the sparsity of the dual Hessian and recast the computation of…
For basic machine learning problems, expected error is used to evaluate model performance. Since the distribution of data is usually unknown, we can make simple hypothesis that the data are sampled independently and identically distributed…
We propose two families of asymptotically local minimax lower bounds on parameter estimation performance. The first family of bounds applies to any convex, symmetric loss function that depends solely on the difference between the estimate…
In this paper we establish lower bounds on information divergence of a distribution on the integers from a Poisson distribution. These lower bounds are tight and in the cases where a rate of convergence in the Law of Thin Numbers can be…