Related papers: Strong data-processing inequalities for channels a…
We develop extreme data compression for use in Bayesian model comparison via the MOPED algorithm, as well as more general score compression. We find that Bayes factors from data compressed with the MOPED algorithm are identical to those…
In 2000, Evans et al. [Eva+00] proved the subadditivity of the mutual information in the broadcasting on tree model with binary vertex labels and symmetric channels. They raised the question of whether such subadditivity extends to loopy…
We provide a transparent, simple and unified treatment of recent results on the equality conditions for the data processing inequality (DPI) of the sandwiched quantum R\'enyi divergence, including the statement that equality in the data…
The deletion channel is the simplest point-to-point communication channel that models lack of synchronization. Despite significant effort, little is known about its capacity, and even less about optimal coding schemes. In this paper we…
We prove a new extremal inequality, motivated by the vector Gaussian broadcast channel and the distributed source coding with a single quadratic distortion constraint problems. As a corollary, this inequality yields a generalization of the…
Many network analysis and graph learning techniques are based on models of random walks which require to infer transition matrices that formalize the underlying stochastic process in an observed graph. For weighted graphs, it is common to…
This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…
We study the exponential dissipation of entropic functionals for continuous time Markov chains and the associated convex Sobolev inequalities, including MLSI and Beckner inequalities. We propose a method that combines the Bakry \'Emery…
We introduce a general algorithm for the computation of the scale functions of a spectrally negative L\'evy process $X$, based on a natural weak approximation of $X$ via upwards skip-free continuous-time Markov chains with stationary…
In image reconstruction, an accurate quantification of uncertainty is of great importance for informed decision making. Here, the Bayesian approach to inverse problems can be used: the image is represented through a random function that…
Modeling uncertainty in deep neural networks, despite recent important advances, is still an open problem. Bayesian neural networks are a powerful solution, where the prior over network weights is a design choice, often a normal…
The fairness of a deep neural network is strongly affected by dataset bias and spurious correlations, both of which are usually present in modern feature-rich and complex visual datasets. Due to the difficulty and variability of the task,…
The sum capacity of a class of discrete memoryless interference channels is determined. This class of channels is defined analogous to the Gaussian Z-interference channel with weak interference; as a result, the sum capacity is achieved by…
Divide-and-conquer Bayesian methods consist of three steps: dividing the data into smaller computationally manageable subsets, running a sampling algorithm in parallel on all the subsets, and combining parameter draws from all the subsets.…
We analytically approximate the expected sum capacity loss between the optimal downlink precoding technique of dirty paper coding (DPC), and the sub-optimal technique of zero-forcing precoding, for multiuser channels. We also consider the…
Despite the remarkable empirical performance of Transformers, their theoretical understanding remains elusive. Here, we consider a deep multi-head self-attention network, that is closely related to Transformers yet analytically tractable.…
We study the following one-way asymmetric transmission problem, also a variant of model-based compressed sensing: a resource-limited encoder has to report a small set $S$ from a universe of $N$ items to a more powerful decoder (server). The…
This paper establishes the exact strong converse exponent of the soft covering problem in the classical setting. This exponent characterizes the slowest achievable convergence speed of the total variation to one when a code of rate below…
The use of parity-check gates in information theory has proved to be very efficient. In particular, error correcting codes based on parity checks over low-density graphs show excellent performances. Another basic issue of information…
Compression and computational efficiency in deep learning have become a problem of great significance. In this work, we argue that the most principled and effective way to attack this problem is by adopting a Bayesian point of view, where…