Related papers: A Generalized Leakage Interpretation of Alpha-Mutu…
We study generalized free fields (GFF) from the point of view of information measures. We first review conformal GFF, their holographic representation, and the ambiguities in the assignation of algebras to regions that arise in these…
We study the generalization error of stochastic learning algorithms from an information-theoretic perspective, with a particular emphasis on deriving sharper bounds for differentially private algorithms. It is well known that the…
In this work, we analyse how to define a conditional version of Sibson's $\alpha$-Mutual Information. Several such definitions can be advanced and they all lead to different information measures with different (but similar) operational…
Pointwise maximal leakage (PML) is an operationally meaningful privacy measure that quantifies the amount of information leaking about a secret $X$ to a single outcome of a related random variable $Y$. In this paper, we extend the notion of…
We introduce a new perspective into the field of quantitative information flow (QIF) analysis that invites the community to bound the leakage, reported by QIF quantifiers, by a range consistent with the size of a program's secret input…
Information theoretic leakage metrics quantify the amount of information about a private random variable $X$ that is leaked through a correlated revealed variable $Y$. They can be used to evaluate the privacy of a system in which an…
Alpha-based performance evaluation may fail to capture correlated residuals due to model errors. This paper proposes using the Generalized Information Ratio (GIR) to measure performance under misspecified benchmarks. Motivated by the…
Recent advances in large language models (LLMs) have revolutionized natural language processing, yet evaluating their intrinsic linguistic understanding remains challenging. Moving beyond specialized evaluation tasks, we propose an…
Federated learning (FL) is a widely adopted privacy-preserving distributed learning framework, yet its generalization performance remains less explored compared to centralized learning. In FL, the generalization error consists of two…
We leverage the Gibbs inequality and its natural generalization to R\'enyi entropies to derive closed-form parametric expressions of the optimal lower bounds of $\rho$th-order guessing entropy (guessing moment) of a secret taking values on…
We discuss a new approach to functional interpretations based on uniform quantification and relativization. The uniform quantification in the background permits a more penetrating analysis of principles related to collection and…
We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI…
A game is introduced to study the effect of privacy in strategic communication between well-informed senders and a receiver. The receiver wants to accurately estimate a random variable. The sender, however, wants to communicate a message…
Taking a functional approach, we derive a general expression for the gradient of the Mutual Information (MI) with respect to the system parameters in the stochastic systems. This expression covers the cases in which the system input depends…
We introduce the study of information leakage through \emph{guesswork}, the minimum expected number of guesses required to guess a random variable. In particular, we define \emph{maximal guesswork leakage} as the multiplicative decrease,…
The quantification and inference of predictive importance for exposure covariates have recently gained significant attention in the context of interpretable machine learning. Contemporary scientific investigations often involve data…
Mutual information (MI) is one of the most general ways to measure relationships between random variables, but estimating this quantity for complex systems is challenging. Denoising diffusion models have recently set a new bar for density…
Vulnerability of Frontier language models to misuse and jailbreaks has prompted the development of safety measures like filters and alignment training in an effort to ensure safety through robustness to adversarially crafted prompts. We…
Deep neural networks obtain state-of-the-art performance on a series of tasks. However, they are easily fooled by adding a small adversarial perturbation to input. The perturbation is often human imperceptible on image data. We observe a…
We introduce a privacy measure called pointwise maximal leakage, generalizing the pre-existing notion of maximal leakage, which quantifies the amount of information leaking about a secret $X$ by disclosing a single outcome of a (randomized)…