相关论文: A general nonconvex large deviation result II

In this paper, we propose a new way to obtain optimal convergence rates for smooth stochastic (strong) convex optimization tasks. Our approach is based on results for optimization tasks where gradients have nonrandom noise. In contrast to…

In this paper, we establish several new inequalities for some twice differantiable mappings. Then, we apply these inequalities to obtain new midpoint, trapezoid and perturbed trapezoid rules. Finally, some applications for special means of…

A rigorous connection between large deviations theory and Gamma-convergence is established. Applications include representations formulas for rate functions, a contraction principle for measurable maps, a large deviations principle for…

We present here a simple method for computing the large deviation of long time average for stochastic jump processes. We show that the computation of the rate function can be reduced to that of a partial differential equation governing the…

The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the existing convergence…

This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting. A class of stochastic momentum methods, including stochastic gradient descent, heavy ball, and Nesterov's accelerated…

I consider general reflection coefficients for arbitrary one-dimensional whole line differential or difference operators of order $2$. These reflection coefficients are semicontinuous functions of the operator: their absolute value can only…

Motivated by policy gradient methods in the context of reinforcement learning, we identify a large deviation rate function for the iterates generated by stochastic gradient descent for possibly non-convex objectives satisfying a…

We discuss a general technique that can be used to form a differentiable bound on the optima of non-differentiable or discrete objective functions. We form a unified description of these methods and consider under which circumstances the…

A large deviation principle is established for a general class of stochastic flows in the small noise limit. This result is then applied to a Bayesian formulation of an image matching problem, and an approximate maximum likelihood property…

We propose a variant of the classical conditional gradient method for sparse inverse problems with differentiable measurement models. Such models arise in many practical problems including superresolution, time-series modeling, and matrix…

We provide an improved analysis of normalized SGD showing that adding momentum provably removes the need for large batch sizes on non-convex objectives. Then, we consider the case of objectives with bounded second derivative and show that…

In this note, we show a sublinear nonergodic convergence rate for the algorithm developed in [Bai, et al. Generalized symmetric ADMM for separable convex optimization. Comput. Optim. Appl. 70, 129-170 (2018)], as well as its linear…

Complex geometries can be easily treated using the well-known full-way and half-way bounce-back rules. However, the accuracy of the full-way bounce-back rule is one order lower than the half-way bounce-back rule. Moreover, when the walls…

In this paper we study second-order optimality conditions for non-convex set-constrained optimization problems. For a convex set-constrained optimization problem, it is well-known that second-order optimality conditions involve the support…

We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are…

This note presents sharp inequalities for deviation probability of a general quadratic form of a random vector \(\xiv\) with finite exponential moments. The obtained deviation bounds are similar to the case of a Gaussian random vector. The…

We investigate large deviations for the empirical measure of the forward and backward recurrence time processes associated with a classical renewal process with arbitrary waiting-time distribution. The Donsker-Varadhan theory cannot be…

Large deviation estimates are by now a standard tool inthe Asymptotic Convex Geometry, contrary to small deviationresults. In this note we present a novel application of a smalldeviations inequality to a problem related to the diameters of…

This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make…