Related papers: Improving Generalization Bounds for VC Classes Usi…

Fix a sequence c=(c_1,...,c_n) of non-negative integers with sum n-1. We say a rooted tree T has child sequence c if it is possible to order the nodes of T as v_1,...,v_n so that for each 1 <= i <= n, v_i has exactly c_i children. Let T be…

Understanding the generalization abilities of modern machine learning algorithms has been a major research topic over the past decades. In recent years, the learning dynamics of Stochastic Gradient Descent (SGD) have been related to…

In recent years, tensors have been applied to different applications in science and engineering fields. In order to establish theory about tail bounds of the tensors summation behavior, this work extends previous work by considering the…

We construct a new tail bound for the sum of independent random variables for situations in which the expected value of the sum is known and each random variable lies within a specified interval, which may be different for each variable.…

Modern regression problems often involve high-dimensional data and a careful tuning of the regularization hyperparameters is crucial to avoid overly complex models that may overfit the training data while guaranteeing desirable properties…

Exponential tail bounds for sums play an important role in statistics, but the example of the $t$-statistic shows that the exponential tail decay may be lost when population parameters need to be estimated from the data. However, it turns…

Multivariate regular variation plays a role assessing tail risk in diverse applications such as finance, telecommunications, insurance and environmental science. The classical theory, being based on an asymptotic model, sometimes leads to…

This paper explores the generalization characteristics of iterative learning algorithms with bounded updates for non-convex loss functions, employing information-theoretic techniques. Our key contribution is a novel bound for the…

Aimed at explaining the surprisingly good generalization behavior of overparameterized deep networks, recent works have developed a variety of generalization bounds for deep learning, all based on the fundamental learning-theoretic…

Many algorithms have been recently proposed for causal machine learning. Yet, there is little to no theory on their quality, especially considering finite samples. In this work, we propose a theory based on generalization bounds that…

The rate of uniform convergence in extreme value statistics is non-universal and can be arbitrarily slow. Further, the relative error can be unbounded in the tail of the approximation, leading to difficulty in extrapolating the extreme…

Generalization in deep learning has been the topic of much recent theoretical and empirical research. Here we introduce desiderata for techniques that predict generalization errors for deep learning models in supervised learning. Such…

This work introduces the minimax Laplace transform method, a modification of the cumulant-based matrix Laplace transform method developed in "User-friendly tail bounds for sums of random matrices" (arXiv:1004.4389v6) that yields both upper…

Recent research has used margin theory to analyze the generalization performance for deep neural networks (DNNs). The existed results are almost based on the spectrally-normalized minimum margin. However, optimizing the minimum margin…

The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm. While many works provide high-probability guarantees, quantifying…

In this note we prove bounds on the upper and lower probability tails of sums of independent geometric or exponentially distributed random variables. We also prove negative results showing that our established tail bounds are asymptotically…

Deep neural networks generalize well despite being heavily overparameterized, in apparent contradiction with classical learning theory based on uniform convergence over fixed hypothesis spaces. Uniform bounds over the entire parameter space…

Our main focus is on the generalization bound, which serves as an upper limit for the generalization error. Our analysis delves into regression and classification tasks separately to ensure a thorough examination. We assume the target…

Hypergraph neural networks have been promising tools for handling learning tasks involving higher-order data, with notable applications in web graphs, such as modeling multi-way hyperlink structures and complex user interactions. Yet, their…

We establish a statistical learning theoretical framework aimed at extrapolation, or out-of-domain generalization, on the unobserved tails of covariates in continuous regression problems. Our strategy involves performing statistical…