Related papers: Central Limit Theorem on Symmetric Kullback-Leible…
In this paper we provide the asymptotic theory of the general of $\phi$-divergences measures, which include the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. We are interested in divergence…
The law of large numbers (LLN) and central limit theorem (CLT) are long and widely been known as two fundamental results in probability theory. Recently problems of model uncertainties in statistics, measures of risk and superhedging in…
In this paper we study uniform versions of two limit theorems in random left truncation model (RLTM). The law of large numbers (LLN) and the central limit theorem (CLT) have been obtained under the bracketing entropy conditions in this…
We consider a random walk $(Y_N)_{N\geq 0}$ on $\mathbb{R}^2$ generated by successively applying independent random isometries, drawn from a fixed measure $\mu$, to the point $0$. When the support of $\mu$ is finite and includes an…
Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we prove several properties of KL divergence between multivariate Gaussian distributions. First, for any two…
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL…
In this paper, we establish the convergence rate in central limit theorem (CLT) for linearly extended negative quadrant dependent (LENQD) random variables (rv's). Under some weak conditions, the rate of normal approximation is shown as…
We prove the central limit theorem (CLT) for a sequence of independent zero-mean random variables $\xi_j$, perturbed by predictable multiplicative factors $\lambda_j$ with values in intervals $[\underline\lambda_j,\overline\lambda_j]$. It…
We give a new proof of the classical Central Limit Theorem, in the Mallows ($L^r$-Wasserstein) distance. Our proof is elementary in the sense that it does not require complex analysis, but rather makes use of a simple subadditive inequality…
Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
We derive the Kullback-Leibler divergence for the normal-gamma distribution and show that it is identical to the Bayesian complexity penalty for the univariate general linear model with conjugate priors. Based on this finding, we provide…
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…
If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically…
We prove two theorems related to the Central Limit Theorem (CLT) for Martin-L\"of Random (MLR) sequences. Martin-L\"of randomness attempts to capture what it means for a sequence of bits to be "truly random". By contrast, CLTs do not make…
Central limit theorems (CLTs) have a long history in probability and statistics. They play a fundamental role in constructing valid statistical inference procedures. Over the last century, various techniques have been developed in…
The "typical" asymptotic behavior of the weighted sums of independent random vectors in $k$-dimensional space is considered. It is shown that in this case the rate of convergence in the multivariate central limit theorem is of order…
The deleting items theorems of weak law of large numbers (WLLN),strong law of large numbers (SLLN) and central limit theorem (CLT) are derived by substituting partial sum of random variable sequence with deleting items partial sum. We…
The aim of this note is to announce some results about the probabilistic and deterministic asymptotic properties of linear groups. The first one is the analogue, for norms of random matrix products, of the classical theorem of Cramer on…
Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with…