Related papers: A general nonconvex large deviation result II

Large-scale nonconvex optimization problems are ubiquitous in modern machine learning, and among practitioners interested in solving them, Stochastic Gradient Descent (SGD) reigns supreme. We revisit the analysis of SGD in the nonconvex…

For sequences of non-lattice weakly dependent random variables, we obtain asymptotic expansions for Large Deviation Principles. These expansions, commonly referred to as strong large deviation results, are in the spirit of Edgeworth…

Several recent works have explored stochastic gradient methods for variational inference that exploit the geometry of the variational-parameter space. However, the theoretical properties of these methods are not well-understood and these…

We obtain large deviations theorems for nonconventional sums with underlying process being a Markov process satisfying the Doeblin condition or a dynamical system such as subshift of finite type or hyperbolic or expanding transformation.

This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…

A new nonparametric estimator of a convex regression function in any dimension is proposed and its convergence properties are studied. We start by using any estimator of the regression function and we \emph{convexify} it by taking the…

This paper studies the convexity properties of nonsmooth extended-real-valued weakly convex functions, a class of functions that is central to modern optimization and its applications. We establish new characterizations of convexity using…

In this paper we analyze several new methods for solving nonconvex optimization problems with the objective function formed as a sum of two terms: one is nonconvex and smooth, and another is convex but simple and its structure is known.…

This study focuses on convex functions and their generalized. Thus, we start this study by giving the definition of convex functions and some of their properties and discussing a simple geometric property. Then we generalize E-convex…

The Large Deviation Principle is established for stochastic models defined by past-dependent non linear recursions with small noise. In the Markov case we use the result to obtain an explicit expression for the asymptotics of exit time.

In these notes we fill a gap in a proof in Section 4 of Gamboa, Nagel, Rouault [Sum rules via large deviations, J. Funct. Anal. 270 (2016), 509-559]. We prove a general theorem which combines a LDP with a convex rate function and a LDP with…

We prove convergence of a single time-scale stochastic subgradient method with subgradient averaging for constrained problems with a nonsmooth and nonconvex objective function having the property of generalized differentiability. As a tool…

An adaptive regularization algorithm for unconstrained nonconvex optimization is proposed that is capable of handling inexact objective-function and derivative values, and also of providing approximate minimizer of arbitrary order. In…

We provide posterior contraction rates for constrained deep Gaussian processes in non-parametric density estimation and classication. The constraints are in the form of bounds on the values and on the derivatives of the Gaussian processes…

We give an extension to a nonconvex setting of the classical radial representation result for lower semicontinuous envelope of a convex function on the boundary of its effective domain. We introduce the concept of radial uniform upper…

We develop the theory of discrete-time gradient flows for convex functions on Alexandrov spaces with arbitrary upper or lower curvature bounds. We employ different resolvent maps in the upper and lower curvature bound cases to construct…

We consider variants of a recently-developed Newton-CG algorithm for nonconvex problems \citep{royer2018newton} in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on…

This paper focuses on systems of nonlinear second-order stochastic differential equations with multi-scales. The motivation for our study stems from mathematical physics and statistical mechanics, for examples, Langevin dynamics and…

Gradient Langevin dynamics and a variety of its variants have attracted increasing attention owing to their convergence towards the global optimal solution, initially in the unconstrained convex framework while recently even in convex…

We prove general nonlinear large deviation estimates similar to Chatterjee-Dembo's original bounds except that we do not require any second order smoothness. Our approach relies on convex analysis arguments and is valid for a broad class of…