Related papers: Generalized weights and bounds for error probabili…

Estimating causal effects from observational data is a central problem in many domains. A general approach is to balance covariates with weights such that the distribution of the data mimics randomization. We present generalized balancing…

This paper studies the novel concept of weight correlation in deep neural networks and discusses its impact on the networks' generalisation ability. For fully-connected layers, the weight correlation is defined as the average cosine…

Upper bounds are given for the weight distribution of binary weakly self-dual codes. To get these new bounds, we introduce a novel method of utilizing unitary operations on Hilbert spaces. This method is motivated by recent progress on…

We derive improved bounds on the error and erasure rate for spherical codes and for binary linear codes under Forney's erasure/list decoding scheme and prove some related results.

We present a general approach, based on exponential inequalities, to derive bounds on the generalization error of randomized learning algorithms. Using this approach, we provide bounds on the average generalization error as well as bounds…

Motivated by iterative decoding techniques for the binary erasure channel Hollmann and Tolhuizen introduced and studied the notion of generic erasure correcting sets for linear codes. A generic $(r,s)$--erasure correcting set generates for…

We propose a unified theory of generalized weights for linear codes endowed with an arbitrary distance. Instead of relying on supports or anticodes, the weights of a code are defined via the intersections of the code with a chosen family of…

We prove new probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifiers. Such combinations could be implemented by neural networks or by voting methods of combining the classifiers,…

A unified framework to obtain all known lower bounds (random coding, typical random coding and expurgated bound) on the reliability function of a point-to-point discrete memoryless channel (DMC) is presented. By using a similar idea for a…

The high computational cost of approaching the performance of Maximum-likelihood (ML) decoding has limited its practical use for decades. Because the complexity grows exponentially with the message length, researchers have spent years…

We obtain new lower and upper bounds for probabilities of unions of events.These bounds are sharp. They are stronger than earlier ones. General bounds maybe applied in arbitrary measurable spaces.We have improved the method that has been…

In a recently published paper [1], it is shown that deep neural networks (DNNs) with random Gaussian weights preserve the metric structure of the data, with the property that the distance shrinks more when the angle between the two data…

Generalized equations are problems emerging in contexts of modern variational analysis as an adequate formalism to treat such issues as constraint systems, optimality and equilibrium conditions, variational inequalities, differential…

The primary objective of learning methods is generalization. Classic uniform generalization bounds, which rely on VC-dimension or Rademacher complexity, fail to explain the significant attribute that over-parameterized models in deep…

This paper considers a multi-source multi-relay network, in which relay nodes employ a coding scheme based on random linear network coding on source packets and generate coded packets. If a destination node collects enough coded packets, it…

It is common practice to collect observations of feature and response pairs from different environments. A natural question is how to identify features that have consistent prediction power across environments. The invariant causal…

This work contains two main contributions concerning the asymmetric broadcast channel. The first is an analysis of the exact random coding error exponents for both users, and the second is the derivation of universal decoders for both…

Tight lower and upper bounds on the ratio of relative entropies of two probability distributions with respect to a common third one are established, where the three distributions are collinear in the standard $(n-1)$-simplex. These bounds…

Many algorithms have been recently proposed for causal machine learning. Yet, there is little to no theory on their quality, especially considering finite samples. In this work, we propose a theory based on generalization bounds that…

In this paper, we leverage stochastic projection and lossy compression to establish new conditional mutual information (CMI) bounds on the generalization error of statistical learning algorithms. It is shown that these bounds are generally…