Related papers: Generalized power cones: optimal error bounds and …
Deep neural network (NN) with millions or billions of parameters can perform really well on unseen data, after being trained from a finite training set. Various prior theories have been developed to explain such excellent ability of NNs,…
PAC-Bayesian bounds are known to be tight and informative when studying the generalization ability of randomized classifiers. However, they require a loose and costly derandomization step when applied to some families of deterministic…
We prove several results concerning automorphism groups of quasismooth complex weighted projective hypersurfaces; these generalize and strengthen existing results for hypersurfaces in ordinary projective space. First, we prove in most cases…
In application areas where data generation is expensive, Gaussian processes are a preferred supervised learning model due to their high data-efficiency. Particularly in model-based control, Gaussian processes allow the derivation of…
Nonconvex optimization is central to modern machine learning, but the general framework of nonconvex optimization yields weak convergence guarantees that are too pessimistic compared to practice. On the other hand, while convexity enables…
In a recent paper~\cite{paper2}, we proposed the concept of optimal error bounds for an iterative process, which allows us to obtain the convergence result of the iterative sequence to the common fixed point of the nonexpansive mappings in…
Using the Moore--Penrose pseudoinverse, this work generalizes the gradient approximation technique called centred simplex gradient to allow sample sets containing any number of points. This approximation technique is called the…
Empirical evidence shows that ensembles, such as bagging, boosting, random and rotation forests, generally perform better in terms of their generalization error than individual classifiers. To explain this performance, Schapire et al.…
Why is it that semidefinite relaxations have been so successful in numerous applications in computer vision and robotics for solving non-convex optimization problems involving rotations? In studying the empirical performance we note that…
We present a universal framework for quantum error-correcting codes, i.e., the one that applies for the most general quantum error-correcting codes. This framework is established on the group algebra, an algebraic notation for the nice…
Conformal prediction (CP) and its extension, conformal risk control (CRC), are established frameworks for quantifying uncertainty in supervised machine learning through formal guarantees. However, recent breakthroughs in artificial…
Length generalization is a key property of a learning algorithm that enables it to make correct predictions on inputs of any length, given finite training data. To provide such a guarantee, one needs to be able to compute a length…
We propose a unifying general framework of quantitative primal and dual sufficient and necessary error bound conditions covering linear and nonlinear, local and global settings. The function is not assumed to possess any particular…
Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a much slower pace. Recent developments have investigated metrics…
Algorithm unfolding or unrolling is the technique of constructing a deep neural network (DNN) from an iterative algorithm. Unrolled DNNs often provide better interpretability and superior empirical performance over standard DNNs in signal…
Generalization error bounds for deep neural networks trained by stochastic gradient descent (SGD) are derived by combining a dynamical control of an appropriate parameter norm and the Rademacher complexity estimate based on parameter norms.…
A general framework for goal-oriented a posteriori error estimation for finite volume methods is presented. The framework does not rely on recasting finite volume methods as special cases of finite element methods, but instead directly…
We study problem-dependent rates, i.e., generalization errors that scale near-optimally with the variance, the effective loss, or the gradient norms evaluated at the "best hypothesis." We introduce a principled framework dubbed "uniform…
Recently proposed generative models for discrete data, such as Masked Diffusion Models (MDMs), exploit conditional independence approximations to reduce the computational cost of popular Auto-Regressive Models (ARMs), at the price of some…
An antinorm is a concave analogue of a norm. In contrast to norms, antinorms are not defined on the entire space $R^d$ but on a cone $K\subset R^d$. They are applied in the matrix analysis, optimal control, and dynamical systems. Their…