Related papers: Lower bounds on Information Divergence

This work obtains sharp closed-form exponential concentration inequalities of Bernstein type for the ubiquitous beta distribution, improving upon sub-gaussian and sub-gamma bounds previously studied in this context. The proof leverages a…

The Collision problem is to decide whether a given list of numbers $(x_1,\ldots,x_n)\in[n]^n$ is $1$-to-$1$ or $2$-to-$1$ when promised one of them is the case. We show an $n^{\Omega(1)}$ randomised communication lower bound for the natural…

In this paper we present various new inequalities for tail proabilities for distributions that are elements of the most improtant exponential families. These families include the Poisson distributions, the Gamma distributions, the binomial…

Percolation in an information-theoretically secure graph is considered where both the legitimate and the eavesdropper nodes are distributed as Poisson point processes. For both the path-loss and the path-loss plus fading model, upper and…

This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make…

In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data…

This paper is an overview of the classical level crossing problem which is studied extensively in the literature and is fundamental in many branches of applied probability. We discuss a number of approximations with an emphasis on their…

For functions of independent random variables, various upper and lower variance bounds are revisited in diverse settings. These are then specialized to the Bernoulli, Gaussian, infinitely divisible cases and to Banach space valued random…

This paper derives lower bounds for the mean square errors of parameter estimators in the case of Poisson distributed data subjected to multiple abrupt changes. Since both change locations (discrete parameters) and parameters of the Poisson…

Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this…

This paper studies probabilistic rates of convergence for consensus+innovations type of algorithms in random, generic networks. For each node, we find a lower and also a family of upper bounds on the large deviations rate function, thus…

An information measure based on fractional partitions of a set is used to derive a general dependence balance inequality for communication. This inequality is used to obtain new upper bounds on reliable and secret rates for multiterminal…

The analysis of panel count data has garnered considerable attention in the literature, leading to the development of multiple statistical techniques. In inferential analysis, most works focus on leveraging estimating equation-based…

Tight bounds for several symmetric divergence measures are derived in terms of the total variation distance. It is shown that each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these…

We develop minimax optimal risk bounds for the general learning task consisting in predicting as well as the best function in a reference set $\mathcal{G}$ up to the smallest possible additive term, called the convergence rate. When the…

This paper presents a stochastic geometry model for the investigation of fundamental information theoretic limitations in wireless networks. We derive a new unified multi-parameter cut-set bound on the capacity of networks of arbitrary…

The sample complexity of estimating or maximising an unknown function in a reproducing kernel Hilbert space is known to be linked to both the effective dimension and the information gain associated with the kernel. While the information…

Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…

We prove an optimal $\Omega(n)$ lower bound on the randomized communication complexity of the much-studied Gap-Hamming-Distance problem. As a consequence, we obtain essentially optimal multi-pass space lower bounds in the data stream model…

We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential…