Related papers: Chain rules for one-shot entropic quantities via o…
The chain rule for the Shannon and von Neumann entropy, which relates the total entropy of a system to the entropies of its parts, is of central importance to information theory. Here we consider the chain rule for the more general smooth…
In this work we derive a number of chain rules for mutual information quantities, suitable for analyzing quantum cryptography with imperfect devices that leak additional information to an adversary. First, we derive a chain rule between…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
One-shot information theory entertains a plethora of entropic quantities, such as the smooth max-divergence, hypothesis testing divergence and information spectrum divergence, that characterize various operational tasks and are used to…
We show that classical chaining bounds on the suprema of random processes in terms of entropy numbers can be systematically improved when the underlying set is convex: the entropy numbers need not be computed for the entire set, but only…
Security proofs in quantum cryptography rely on conditional entropies. In a many-round protocol, their estimation is a challenging task; one must account for the most general attacks by an eavesdropper, including those that are not…
We study formal properties of smooth max-information, a generalization of von Neumann mutual information derived from the max-relative entropy. Recent work suggests that it is a useful quantity in one-shot channel coding, quantum rate…
Given two orthornormal bases A and B, the basic form of the entropic uncertainty principle is stated in terms of the sum of the Shannon entropies of the probabilities of measuring A and B onto a given quantum state. State independent lower…
We consider two fundamental tasks in quantum information theory, data compression with quantum side information as well as randomness extraction against quantum side information. We characterize these tasks for general sources using…
We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural…
The von Neumann entropy of an $n$-partite system $A_1^n$ given a system $B$ can be written as the sum of the von Neumann entropies of the individual subsystems $A_k$ given $A_1^{k-1}$ and $B$. While it is known that such a chain rule does…
We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
For discretisations of hyperbolic conservation laws, mimicking properties of operators or solutions at the continuous (differential equation) level discretely has resulted in several successful methods. While well-posedness for nonlinear…
We review several competing chaining methods to estimate the supremum, the diameter of the range or the modulus of continuity of a stochastic process in terms of tail bounds of their two-dimensional distributions. Then we show how they can…
Many modern numerical methods in computational science and engineering rely on derivatives of mathematical models for the phenomena under investigation. The computation of these derivatives often represents the bottleneck in terms of…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
While deep learning approaches to information extraction have had many successes, they can be difficult to augment or maintain as needs shift. Rule-based methods, on the other hand, can be more easily modified. However, crafting rules…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…