Related papers: Generalized weights and bounds for error probabili…
We present a new family of information-theoretic generalization bounds within the framework of conditional mutual information (CMI). Most of our results are established based on the leave-$m$-out (L$m$O) cross-validation error, with $m$…
Usually standard algorithms employ a loss where each error is the mere absolute difference between the true value and the prediction, in case of a regression task. In the present, we introduce several error weighting schemes that are a…
In this letter, we analyse the properties of a maximum likelihood channel estimator based on the syndrome of a linear code. For the two examples of a binary symmetric channel and a binary input additive white Gaussian noise channel, we…
Csisz\'ar's channel coding theorem for multiple codebooks is generalized allowing the codeword lenghts differ across codebooks. Also in this case, for each codebook an error exponent can be achieved that equals the random coding exponent…
In this paper, we derive the exact input/output transfer functions of the optimal a-posteriori probability channel detector for a general ISI channel with erasures. Considering three channel impulse responses of different memory as an…
An erasure channel with a fixed alphabet size $q$, where $q \gg 1$, is studied. It is proved that over any erasure channel (with or without memory), Maximum Distance Separable (MDS) codes achieve the minimum probability of error (assuming…
This paper explores the generalization characteristics of iterative learning algorithms with bounded updates for non-convex loss functions, employing information-theoretic techniques. Our key contribution is a novel bound for the…
Memoryless channels with deletion errors as defined by a stochastic channel matrix allowing for bit drop outs are considered in which transmitted bits are either independently deleted with probability $d$ or unchanged with probability…
Properties of weighted averages are studied for the general case that the individual measurements are subject to hidden correlations and have asymmetric statistical as well as systematic errors. Explicit expressions are derived for an…
We provide a general framework for bounding the block error threshold of a linear code $C\subseteq \mathbb{F}_2^N$ over the erasure channel in terms of its bit error threshold. Our approach relies on understanding the minimum support weight…
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian…
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely…
When information is to be transmitted over an unknown, possibly unreliable channel, an erasure option at the decoder is desirable. Using constant-composition random codes, we propose a generalization of Csiszar and Korner's Maximum Mutual…
We consider the additive white Gaussian noise channels. We prove that the error probability of decoding tends to one exponentially for rates above the capacity and derive the optimal exponent function. We shall demonstrate that the…
Worst-case models of erasure and symmetric channels are investigated, in which the number of channel errors occurring in each sliding window of a given length is bounded. Upper and lower bounds on their zero-error capacities are derived,…
We derive a new theoretical interpretation of the reweighted losses that are widely used for training diffusion models. Our method is based on constructing a cascade of time-dependent variational lower bounds on the data log-likelihood,…
We study relationships between worst-case and random-noise properties of error correcting codes. More concretely, we consider connections between minimum distance, list decoding radius, and block error probability on noisy channels. A…
In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two…
A bilateral (i.e., upper and lower) bound on the mean-square error under a general model mismatch is developed. The bound, which is derived from the variational representation of the chi-square divergence, is applicable in the Bayesian and…
Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing…