相关论文: Normalized Gradients for All
A novel method for segmenting bright objects from dark background for grayscale image is proposed. The concept of this method can be stated simply as: to pick out the local-thinnest bands on the grayscale grade-map. It turns out to be a…
In this paper, we propose a new covering technique localized for the trajectories of SGD. This localization provides an algorithm-specific complexity measured by the covering number, which can have dimension-independent cardinality in…
We develop a novel framework to study smooth and strongly convex optimization algorithms, both deterministic and stochastic. Focusing on quadratic functions we are able to examine optimization algorithms as a recursive application of linear…
Gradient-variation online learning aims to achieve regret guarantees that scale with variations in the gradients of online functions, which has been shown to be crucial for attaining fast convergence in games and robustness in stochastic…
Proving algorithm-dependent generalization error bounds for gradient-type optimization methods has attracted significant attention recently in learning theory. However, most existing trajectory-based analyses require either restrictive…
Random smoothing data augmentation is a unique form of regularization that can prevent overfitting by introducing noise to the input data, encouraging the model to learn more generalized features. Despite its success in various…
Smoothness of generalized solutions for higher-order elliptic equations with nonlocal boundary conditions is studied in plane domains. Necessary and sufficient conditions upon the right-hand side of the problem and nonlocal operators under…
A new paradigm to estimate the gradient of a black-box scalar function is introduced, considering it as a member of a set of admissible gradients that are computed using existing function samples. Results on gradient estimate accuracy,…
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In this article, we study these types of methods through the…
An influential line of recent work has focused on the generalization properties of unregularized gradient-based learning procedures applied to separable linear classification with exponentially-tailed loss functions. The ability of such…
We wish to compute the gradient of an expectation over a finite or countably infinite sample space having $K \leq \infty$ categories. When $K$ is indeed infinite, or finite but very large, the relevant summation is intractable. Accordingly,…
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient. This generally requires an expensive hyperparameter tuning process to appropriately calibrate a stepsize for a given…
In this paper we shall establish some regularity results of solutions of a class of fully nonlinear equations, with a first order term which is sub-linear. We prove local H\"older regularity of the gradient both in the interior and up to…
This paper proposes hybrid high-order eigensolvers for the computation of guaranteed lower eigenvalue bounds. These bounds display higher order convergence rates and are accessible to adaptive mesh-refining algorithms. The involved…
We establish novel generalization bounds for learning algorithms that converge to global minima. We do so by deriving black-box stability results that only depend on the convergence of a learning algorithm and the geometry around the…
We provide a theoretical explanation for the effectiveness of gradient clipping in training deep neural networks. The key ingredient is a new smoothness condition derived from practical neural network training examples. We observe that…
We study smoothness of generalized solutions of nonlocal elliptic problems in plane bounded domains with piecewise smooth boundary. The case where the support of nonlocal terms can intersect the boundary is considered. We find conditions…
We propose a new framework for black-box convex optimization which is well-suited for situations where gradient computations are expensive. We derive a new method for this framework which leverages several concepts from convex optimization,…
We present an optimal gradient method for smooth strongly convex optimization. The method is optimal in the sense that its worst-case bound on the distance to an optimal point exactly matches the lower bound on the oracle complexity for the…
In this paper, we derive global bounds for the H\"older norm of the gradient of solutions of graphic mean curvature flow with boundary of arbitrary codimension.