Related papers: Strong Asymptotic Composition Theorems for Mutual …
A general method is presented for deriving the limiting behavior of estimators that are defined as the values of parameters optimizing an empirical criterion function. The asymptotic behavior of such estimators is typically deduced from…
We introduce a family of information leakage measures called maximal $\alpha,\beta$-leakage, parameterized by real numbers $\alpha$ and $\beta$. The measure is formalized via an operational definition involving an adversary guessing an…
The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted…
A conditional version of Sibson's $\alpha$-information is defined using a simple closed-form "log-expectation" expression, which satisfies important properties such as consistency, uniform expansion, and data processing inequalities. This…
We consider the problem of quantifying the information shared by a pair of random variables $X_{1},X_{2}$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left…
In the study of extremes, the presence of asymptotic independence signifies that extreme events across multiple variables are probably less likely to occur together. Although well-understood in a bivariate context, the concept remains…
A common countermeasure against side-channel attacks on secret key cryptographic implementations is $d$th-order masking, which splits each sensitive variable into $d+1$ random shares. In this paper, maximal leakage bounds on the probability…
Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines,…
We establish the Strong Poisson Hypothesis for symmetric closed networks. In particular, the asymptotic independence of the nodes -- as the size of the system tends to infinity -- is proved.
Aggregation of noisy observations involves a difficult tradeoff between observation quality, which can be increased by increasing the number of observations, and aggregation quality which decreases if the number of observations is too…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
We derive an asymptotic expansion for the distribution of a compound sum of independent random variables, all having the same light-tailed subexponential distribution. The examples of a Poisson and geometric number of summands serve as an…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
For $\tilde{f}(t) = \exp(\frac{\alpha-1}{\alpha}t)$, this paper shows that the Sibson mutual information is an $\alpha$-leakage averaged over the adversary's $\tilde{f}$-mean relative information gain (on the secret) at elementary event of…
In this work, maximal $\alpha$-leakage is introduced to quantify how much a quantum adversary can learn about any sensitive information of data upon observing its disturbed version via a quantum privacy mechanism. We first show that an…
Does the asymptotic variance of the maximum composite likelihood estimator of a parameter of interest always decrease when the nuisance parameters are known? Will a composite likelihood necessarily become more efficient by incorporating…
Random measures provide flexible parameters for Bayesian nonparametric models. Given two different priors for a random measure, we develop a natural framework to investigate the rate at which the corresponding posteriors merge, as the…
The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…
In this paper we improve the probabilistic approach to compositions of Ehrenborg, Levin and Readdy by introducing a simpler but more general probabilistic model. As consequence we get some new estimates on the behavior of a uniform random…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…