Related papers: Information-Theoretic Perspectives on Brascamp-Lie…
We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information…
It is known that by dualizing the Bochner-Lichnerowicz-Weitzenb\"{o}ck formula, one obtains Poincar\'e-type inequalities on Riemannian manifolds equipped with a density, which satisfy the Bakry-\'Emery Curvature-Dimension condition…
In the context of statistical learning, the Information Bottleneck method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description…
The aim of this note is to connect a reversed form of the Gross logarithmic Sobolev inequality with the Gaussian maximum of Shannon's entropy power. There is thus a complete parallel with the well-known link between logarithmic Sobolev…
It was observed recently in work of Bez, Buschenhenke, Cowling, Flock and the first author, that the euclidean Brascamp-Lieb inequality satisfies a natural and useful Fourier duality property. The purpose of this paper is to establish an…
Lieb and Ruskai's strong subadditivity theorem, which shows that the conditional mutual information must be nonnegative, is fundamental in quantum theory. It has numerous applications, such as in quantum error correction. When the mutual…
This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate…
This paper first focuses on deriving an alternative approach for proving an extremal entropy inequality (EEI), originally presented in [11]. The proposed approach does not rely on the channel enhancement technique, and has the advantage…
Rooted trees with probabilities are convenient to represent a class of random processes with memory. They allow to describe and analyze variable length codes for data compression and distribution matching. In this work, the Leaf-Average…
Information theoretic active learning has been widely studied for probabilistic models. For simple regression an optimal myopic policy is easily tractable. However, for other tasks and with more complex models, such as classification with…
Prior information about the input state can be utilized to enhance the efficiency of quantum teleportation which we quantify using the first two moments of fidelity. The input knowledge is introduced by relaxing the uniformity assumption in…
We initiate the study of a duality theory which applies to norm inequalities for pointwise weighted geometric means of positive operators. The theory finds its expression in terms of certain pointwise factorisation properties of function…
We extend the result of Ball and Nguyen on the jump of entropy under convolution for logconcave random vectors. We show that the result holds for any pair of vectors (not necessarily identically distributed) and that a similar inequality…
We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative…
In this work, we derive information-theoretic properties for a modified Tsallis entropy, hereinafter referred to as q-entropy. We introduce the notions of joint q-entropy, conditional q-entropy, relative q-entropy, conditional mutual…
The information bottleneck (IB) method seeks a compressed representation of data that preserves information relevant to a target variable for prediction while discarding irrelevant information from the original data. In its classical…
We establish fundamental connections between utility theories of wealth from the economic sciences and information-theoretic quantities. In particular, we introduce operational tasks based on betting where both gambler and bookmaker have…
By extracting task-relevant information while maximally compressing the input, the information bottleneck (IB) principle has provided a guideline for learning effective and robust representations of the target inference. However, extending…
Recent investigations into the physical nature of information and fundamental limits to information transmission have revealed questions such as the possibility of superluminal data transfer or not; and whether reversible computation…
This chapter provides a comprehensive and self-contained discussion of the most recent developments of information theory of networks. Maximum entropy models of networks are the least biased ensembles enforcing a set of constraints and are…