相关论文: Limit distribution theory for $f$-Divergences

We study limit theorems for entropic optimal transport (EOT) maps, dual potentials, and the Sinkhorn divergence. The key technical tool we use is a first and second-order Hadamard differentiability analysis of EOT potentials with respect to…

We present a series of new and more favorable margin-based learning guarantees that depend on the empirical margin loss of a predictor. We give two types of learning bounds, both distribution-dependent and valid for general families, in…

The distinguishability quantified by information measures after being processed by a private mechanism has been a useful tool in studying various statistical and operational tasks while ensuring privacy. To this end, standard…

For an unknown continuous distribution on a real line, we consider the approximate estimation by the discretization. There are two methods for the discretization. First method is to divide the real line into several intervals before taking…

Density ratio estimation (DRE) is a core technique in machine learning used to capture relationships between two probability distributions. $f$-divergence loss functions, which are derived from variational representations of $f$-divergence,…

We examine the privacy amplification of channels that do not necessarily satisfy any LDP guarantee by analyzing their contraction behavior in terms of $f_\alpha$-divergence, an $f$-divergence related to R\'enyi-divergence via a monotonic…

This paper addresses the problem of distributed detection in fixed and switching networks. A network of agents observe partially informative signals about the unknown state of the world. Hence, they collaborate with each other to identify…

Estimating the probability distribution 'q' governing the behaviour of a certain variable by sampling its value a finite number of times most typically involves an error. Successive measurements allow the construction of a histogram, or…

We extend the celebrated Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, from its standard setting of total variation distance to all $f$-divergences. A key obstacle in this endeavor is to define…

This tutorial studies relationships between differential privacy and various information-theoretic measures by using several selective articles. In particular, we present how these connections can provide new interpretations for the privacy…

We study statistical estimation under local differential privacy (LDP) when users may hold heterogeneous privacy levels and accuracy must be guaranteed with high probability. Departing from the common in-expectation analyses, and for…

Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and…

This short note is on a property of the Kullback-Leibler (KL) divergence which indicates that independent Gaussian distributions minimize the KL divergence from given independent Gaussian distributions. The primary purpose of this note is…

We consider the problem of estimating probability density functions based on sample data, using a finite mixture of densities from some component class. To this end, we introduce the $h$-lifted Kullback--Leibler (KL) divergence as a…

The classical information metric provides a unique notion of distance on the space of probability distributions with a well-defined operational interpretation: two distributions are far apart if they are readily distinguishable from one…

An invariant ensemble of $N\times N$ random matrices can be characterised by a joint distribution for eigenvalues $P(\lambda_1,\cdots,\lambda_N)$. The study of the distribution of linear statistics, i.e. of quantities of the form…

The most effective differentially private machine learning algorithms in practice rely on an additional source of purportedly public data. This paradigm is most interesting when the two sources combine to be more than the sum of their…

Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification. In this paper, we develop a method for comparing how close different generative…

We present some product representations for random variables with the Linnik, Mittag-Leffler and Weibull distributions and establish the relationship between the mixing distributions in these representations. Based on these representations,…

Diffusion models for continuous state spaces based on Gaussian noising processes are now relatively well understood from both practical and theoretical perspectives. In contrast, results for diffusion models on discrete state spaces remain…