Related papers: Data processing inequalities based on a certain st…
Classification margins are commonly used to estimate the generalization ability of machine learning models. We present an empirical study of these margins in artificial neural networks. A global estimate of margin size is usually used in…
Classical probabilistic models of (noisy) quantum systems are not only relevant for understanding the non-classical features of quantum mechanics, but they are also useful for determining the possible advantage of using quantum resources…
The goal of this paper is to explore the basic Approximate Bayesian Computation (ABC) algorithm via the lens of information theory. ABC is a widely used algorithm in cases where the likelihood of the data is hard to work with or…
One of the most studied problems in machine learning is finding reasonable constraints that guarantee the generalization of a learning algorithm. These constraints are usually expressed as some simplicity assumptions on the target. For…
Consensus is a common method for computing a function of the data distributed among the nodes of a network. Of particular interest is distributed average consensus, whereby the nodes iteratively compute the sample average of the data stored…
The Gromov-Wasserstein (GW) variant of optimal transport, designed to compare probability densities defined over distinct metric spaces, has emerged as an important tool for the analysis of data with complex structure, such as ensembles of…
I consider the tradeoff between the information gained about an initially unknown quantum state, and the disturbance caused to that state by the measurement process. I show that for any distribution of initial states, the…
A continuous-time white Gaussian channel can be formulated using a white Gaussian noise, and a conventional way for examining such a channel is the sampling approach based on the Shannon-Nyquist sampling theorem, where the original…
We introduce an alternative closed form lower bound on the Gaussian process ($\mathcal{GP}$) likelihood based on the R\'enyi $\alpha$-divergence. This new lower bound can be viewed as a convex combination of the Nystr\"om approximation and…
In this correspondence, we correct an erroneous result on the achievability part of the R\'enyi common information with order $1+s\in(1,2]$ in [1]. The new achievability result (upper bound) of the R\'enyi common information no longer…
We consider a new formulation of a class of synchronization error channels and derive analytical bounds and numerical estimates for the capacity of these channels. For the binary channel with only deletions, we obtain an expression for the…
This article considers the challenge of accommodating outlier measurements in state estimation. The Risk-Averse Performance-Specified (RAPS) state estimation approach addresses outliers as a measurement selection Bayesian risk minimization…
The problem of determining the intrinsic quality of a signal processing system with respect to the inference of an unknown deterministic parameter $\theta$ is considered. While the Fisher information measure $F(\theta)$ forms a classical…
A general divergence measure for monotonic functions is introduced. Its connections with the f-divergence for convex functions are explored. The main properties are pointed out.
The radiological characterization of contaminated elements (walls, grounds, objects) from nuclear facilities often suffers from a too small number of measurements. In order to determine risk prediction bounds on the level of contamination,…
We consider a continuous-time bandlimited additive white Gaussian noise channel with 1-bit output quantization. On such a channel the information is carried by the temporal distances of the zero-crossings of the transmit signal. The set of…
Simultaneous transmission of information and power over a point-to-point flat-fading complex Additive White Gaussian Noise (AWGN) channel is studied. In contrast with the literature that relies on an inaccurate linear model of the energy…
Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also…
We consider a classical problem in choice theory -- vote aggregation -- using novel distance measures between permutations that arise in several practical applications. The distance measures are derived through an axiomatic approach, taking…
We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we…