Related papers: Optimal Dimension-Free Sampling for Regularized Cl…

Diffusion generative models have emerged as powerful tools for producing synthetic data from an empirically observed distribution. A common approach involves simulating the time-reversal of an Ornstein-Uhlenbeck (OU) process initialized at…

In this work, we develop first-order (Hessian-free) and zero-order (derivative-free) implementations of the Cubically regularized Newton method for solving general non-convex optimization problems. For that, we employ finite difference…

We study the recovery of functions in various norms, including $L_p$ with $1\le p\le\infty$, based on function evaluations. We obtain worst case error bounds for general classes of functions in terms of the best $L_2$-approximation from a…

We study unconstrained Online Linear Optimization with Lipschitz losses. Motivated by the pursuit of instance optimality, we propose a new algorithm that simultaneously achieves ($i$) the AdaGrad-style second order gradient adaptivity; and…

We prove a sampling discretization theorem for the square norm of functions from a finite dimensional subspace satisfying Nikol'skii's inequality with an upper bound on the number of sampling points of the order of the dimension of the…

We obtain new sampling discretization results in Orlicz norms on finite dimensional spaces. As applications, we study sampling recovery problems, where the error of the recovery process is calculated with respect to different Orlicz norms.…

We study randomized algorithms for constrained optimization, in abstract frameworks that include, in strictly increasing generality: convex programming; LP-type problems; violator spaces; and a setting we introduce, consistent spaces. Such…

We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau…

We study the complexity of optimizing highly smooth convex functions. For a positive integer $p$, we want to find an $\epsilon$-approximate minimum of a convex function $f$, given oracle access to the function and its first $p$ derivatives,…

We prove regularity results for the unique minimizer of the total variation functional, currently used in image processing analysis since the work by L. Rudin, S. Osher and E. Fatemi. In particular we show that if the source term $f$ is…

We give a fast algorithm for sampling uniform solutions of general constraint satisfaction problems (CSPs) in a local lemma regime. Suppose that the CSP has $n$ variables with domain size at most q, each constraint contains at most k…

Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks. Since such bounds often hold uniformly over all parameters, they suffer from over-parametrization and fail to account…

We derive fundamental sampling bounds for smooth signals in continuous settings without sparsity assumptions. By introducing the Fourier ratio as a measure of spectral compressibility induced by smoothness, we obtain explicit, deterministic…

We obtain estimation error rates and sharp oracle inequalities for regularization procedures of the form \begin{equation*} \hat f \in argmin_{f\in F}\left(\frac{1}{N}\sum_{i=1}^N\ell(f(X_i), Y_i)+\lambda \|f\|\right) \end{equation*} when…

We provide a new upper bound for sampling numbers $(g_n)_{n\in \mathbb{N}}$ associated to the compact embedding of a separable reproducing kernel Hilbert space into the space of square integrable functions. There are universal constants…

We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation,…

Robustness with respect to weight perturbations underpins guarantees for generalization, pruning and quantization. Existing guarantees rely on Lipschitz bounds in parameter space, cover only plain feed-forward MLPs, and break under the…

Most current sampling algorithms for high-dimensional distributions are based on MCMC techniques and are approximate in the sense that they are valid only asymptotically. Rejection sampling, on the other hand, produces valid samples, but is…

We study the theoretical properties of random Fourier features classification with Lipschitz continuous loss functions such as support vector machine and logistic regression. Utilizing the regularity condition, we show for the first time…

Let $\mathbb{T}^d$ denote the $d$-dimensional torus. We consider the problem of optimally recovering a target function $f^*:\mathbb{T}^d\rightarrow \mathbb{C}$ from samples of its Fourier coefficients. We make classical smoothness…