Related papers: An Operational Approach to Information Leakage
It is often necessary to disclose training data to the public domain, while protecting privacy of certain sensitive labels. We use information theoretic measures to develop such privacy preserving data disclosure mechanisms. Our mechanism…
The Shannon entropy of a random variable $X$ has much behaviour analogous to a signed measure. Previous work has concretized this connection by defining a signed measure $\mu$ on an abstract information space $\tilde{X}$, which is taken to…
Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion.…
We present quantitative relations between work and information that are valid both for finite sized and internally correlated systems as well in the thermodynamical limit. We suggest work extraction should be viewed as a game where the…
In this work, we present an alternative to conventional residual connections, which is inspired by maxout nets. This means that instead of the addition in residual connections, our approach only propagates the maximum value or, in the leaky…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Two families of dependence measures between random variables are introduced. They are based on the R\'enyi divergence of order $\alpha$ and the relative $\alpha$-entropy, respectively, and both dependence measures reduce to Shannon's mutual…
A measurement performed on a quantum system is an act of gaining information about its state, a view that is widespread in practical and foundational work in quantum theory. However, the concept of information in quantum theory…
Underestimating the leakage can compromise secrecy, while overestimating it may lead to inefficient system design. Therefore, a reliable leakage estimator is essential. Neural network-based estimators provide a data-driven way to estimate…
The total variation distance is proposed as a privacy measure in an information disclosure scenario when the goal is to reveal some information about available data in return of utility, while retaining the privacy of certain sensitive…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
A central question in information theory is to determine the maximum success probability that can be achieved in sending a fixed number of messages over a noisy channel. This was first studied in the pioneering work of Shannon who…
The concept of Relative Divergence of one Grading Function from another is extended from totally ordered chains to power sets of finite event spaces. Shannon Entropy concept is extended to normalized grading functions on such power sets.…
Leakage of data from publicly available Machine Learning (ML) models is an area of growing significance as commercial and government applications of ML can draw on multiple sources of data, potentially including users' and clients'…
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…
This paper technically explores the secrecy rate $\Lambda$ and a maximisation problem over the concave version of the secrecy outage probability (SOP) as $\mathop{{\rm \mathbb{M}ax}}\limits_{\Delta } {\rm \; } \mathbb{P}\mathscr{r} \big(…
An information theoretic privacy mechanism design problem for two scenarios is studied where the private data is either hidden or observable. In each scenario, privacy leakage constraints are considered using two different measures. In…
This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes the pragmatic elements in the derivation. An epistemic notion of information is defined in…
Barycentric and pairwise quantum Renyi leakages are proposed as two measures of information leakage for privacy and security analysis in quantum computing and communication systems. These quantities both require minimal assumptions on the…
We study a statistical signal processing privacy problem, where an agent observes useful data $Y$ and wants to reveal the information to a user. Since the useful data is correlated with the private data $X$, the agent employs a privacy…