Related papers: New Upper bounds for KL-divergence Based on Integr…
Given any finite direction set $\Omega$ of cardinality $N$ in Euclidean space, we consider the maximal directional Hilbert transform $H_{\Omega}$ associated to this direction set. Our main result provides an essentially sharp uniform bound,…
In this technical report, we consider conditional density estimation with a maximum likelihood approach. Under weak assumptions, we obtain a theoretical bound for a Kullback-Leibler type loss for a single model maximum likelihood estimate.…
The law of large numbers for the empirical density for the pairs of uniformly distributed integers with a given greatest common divisor is a classic result in number theory. In this paper, we study the large deviations of the empirical…
The irregularities of a distribution of $N$ points in the unit interval are often measured with various notions of discrepancy. The discrepancy function can be defined with respect to intervals of the form $[0,t)\subset [0,1)$ or arbitrary…
Information generating functions have been used for generating various entropy and divergence measures. In the present work, we introduce quantile based relative information generating function and study its properties. The proposed…
We study the approximation of arbitrary distributions $P$ on $d$-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback--Leibler-type functional. We show that such an approximation exists if…
The L_2-discrepancy measures the irregularity of the distribution of a finite point set. In this note we prove lower bounds for the L_2 discrepancy of arbitrary N-point sets. Our main focus is on the two-dimensional case. Asymptotic upper…
A great challenge in the analysis of the discrepancy function D_N is to obtain universal lower bounds on the L-infty norm of D_N in dimensions d \geq 3. It follows from the average case bound of Klaus Roth that the L-infty norm of D_N is at…
Bayesian inference requires approximation methods to become computable, but for most of them it is impossible to quantify how close the approximation is to the true posterior. In this work, we present a theorem upper-bounding the KL…
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on…
In this paper, we study Lebesgue differentiation processes along rectangles $R_k$ shrinking to the origin in the Euclidean plane, and the question of their almost everywhere convergence in $L^p$ spaces. In particular, classes of examples of…
We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…
Extreme Value Theory plays an important role to provide approximation results for the extremes of a sequence of independent random variables when their distribution is unknown. An important one is given by the {generalised Pareto…
Convergence rates of kernel density estimators for stationary time series are well studied. For invertible linear processes, we construct a new density estimator that converges, in the supremum norm, at the better, parametric, rate…
Multi-dimensional distributions whose marginal distributions are uniform are called copulas. Among them, the one that satisfies given constraints on expectation and is closest to the independent distribution in the sense of Kullback-Leibler…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
This paper investigates the asymptotics of the maximal throughput of communication over AWGN channels by $n$ channel uses under a covert constraint in terms of an upper bound $\delta$ of Kullback-Leibler divergence (KL divergence). It is…
Gradient flows of the Kullback--Leibler (KL) divergence, such as the Fokker--Planck equation and Stein Variational Gradient Descent, evolve a distribution toward a target density known only up to a normalizing constant. We introduce new…
In this paper we provide the asymptotic theory of the general of $\phi$-divergences measures, which includes the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. Instead of using the Parzen…
The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis…