Related papers: Asymptotic Divergences and Strong Dichotomy
For a fixed alphabet $A$, an infinite sequence $X$ is said to be normal if every word $w$ over $A$ appears in $X$ with the same frequency as any other word of the same length. A classical result of Agafonov (1966) relates normality to…
If $S$ is an infinite sequence over a finite alphabet $\Sigma$ and $\beta$ is a probability measure on $\Sigma$, then the {\it dimension} of $ S$ with respect to $\beta$, written $\dim^\beta(S)$, is a constructive version of Billingsley…
If $S$ is an infinite sequence over a finite alphabet $\Sigma$ and $\beta$ is a probability measure on $\Sigma$, then the {\it dimension} of $ S$ with respect to $\beta$, written $\dim^\beta(S)$, is a constructive version of Billingsley…
An infinite sequence $\alpha$ over an alphabet $\Sigma$ is $\mu$-distributed w.r.t. a probability map $\mu$ if, for every finite string $w$, the limiting frequency of $w$ in $\alpha$ exists and equals $\mu(w)$. %We raise the question of how…
A tournament on a graph is an orientation of its edges. The score sequence lists the in-degrees in non-decreasing order. Works by Winston and Kleitman (1983) and Kim and Pittel (2000) showed that the number $S_n$ of score sequences on the…
For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $\alpha$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $\chi^2$…
In this paper we provide the asymptotic theory of the general of $\phi$-divergences measures, which include the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. We are interested in divergence…
For a regression model, we consider the risk of the maximum likelihood estimator with respect to $\alpha$-divergence, which includes the special cases of Kullback-Leibler divergence, Hellinger distance and $\chi^2$ divergence. The…
We consider the sequential composite binary hypothesis testing problem in which one of the hypotheses is governed by a single distribution while the other is governed by a family of distributions whose parameters belong to a known set…
Statistical distances (SDs), which quantify the dissimilarity between probability distributions, are central to machine learning and statistics. A modern method for estimating such distances from data relies on parametrizing a variational…
Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with…
In this paper we provide an asymptotic theory for the symmetric version of the Kullback--Leibler (KL) divergence. We define a estimator for this divergence and study its asymptotic properties. In particular, we prove Law of Large Numbers…
In this paper we provide the asymptotic theory of the general of $\phi$-divergences measures, which includes the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. Instead of using the Parzen…
We study the problem of estimating a distribution over a finite alphabet from an i.i.d. sample, with accuracy measured in relative entropy (Kullback-Leibler divergence). While optimal bounds on the expected risk are known, high-probability…
First, we consider the problem of hedging in complete binomial models. Using the discrete-time F\"ollmer-Schweizer decomposition, we demonstrate the equivalence of the backward induction and sequential regression approaches. Second, in…
We consider the multiple hypothesis testing problem for symmetric quantum state discrimination between r given states \sigma_1,...,\sigma_r. By splitting up the overall test into multiple binary tests in various ways we obtain a number of…
We show that the moment generating function of the Kullback-Leibler divergence (relative entropy) between the empirical distribution of $n$ independent samples from a distribution $P$ over a finite alphabet of size $k$ (i.e. a multinomial…
Let $\boldsymbol W=\{\boldsymbol W_n:n\in\mathbb N\}$ be a sequence of random vectors in $\mathbb R^d$, $d\ge 1$. This paper considers the logarithmic asymptotics of the extremes of $\boldsymbol W$, that is, for any vector $\boldsymbol…
In nonparametric statistical problems, we wish to find an estimator of an unknown function f. We can split its error into bias and variance terms; Smirnov, Bickel and Rosenblatt have shown that, for a histogram or kernel estimate, the…
This paper combines the decomposition technique ($\sigma$-stability) in random functional analysis with the deterministic theory of asymptotically pointwise contractions to provide a complete self-contained derivation of a fixed point…