Related papers: A general nonconvex large deviation result II

The paper is devoted to a detailed analysis of nonlocal error bounds for nonconvex piecewise affine functions. We both improve some existing results on error bounds for such functions and present completely new necessary and/or sufficient…

We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that…

We obtain the exact large deviation functions of the density profile and of the current, in the non-equilibrium steady state of a one dimensional symmetric simple exclusion process coupled to boundary reservoirs with slow rates. Compared to…

In this paper, we propose an inexact Newton-like conditional gradient method for solving constrained systems of nonlinear equations. The local convergence of the new method as well as results on its rate are established by using a general…

We here adapt an extended version of the adaptive cubic regularisation method with dynamic inexact Hessian information for nonconvex optimisation in [3] to the stochastic optimisation setting. While exact function evaluations are still…

In the first part of this study, a convex-constrained penalized formulation was studied for a class of constant modulus (CM) problems. In particular, the error bound techniques were shown to play a vital role in providing exact penalization…

We review convergence and behavior of stochastic gradient descent for convex and nonconvex optimization, establishing various conditions for convergence to zero of the variance of the gradient of the objective function, and presenting a…

This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our…

For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate various approximations. In particular, in addition to approximate…

We obtain a new bound on the second moment of modified shifted convolutions of the generalized 3-fold divisor function, and show that, for applications, the modified version is sufficient.

In this paper, we introduce a new second-order directional derivative and a second-order subdifferential of Hadamard type for an arbitrary nondifferentiable function. We derive several second-order optimality conditions for a local and a…

Regression models with both high-dimensional responses and covariates have attracted growing attention. Standard multivariate regression models become inadequate when the response variables depend not only on observed covariates but also on…

We prove large and moderate deviation principles for the distribution of an empirical mean conditioned by the value of the sum of discrete i.i.d. random variables. Some applications for combinatoric problems are discussed.

We consider a jump-diffusion process on a bounded domain with reflection at the boundary, and establish long-term results for a general additive process of its path. This includes the long-term behaviour of its occupation time in the…

We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theoretical lower bounds and prove that our algorithm achieves…

We develop two novel stochastic variance-reduction methods to approximate solutions of a class of nonmonotone [generalized] equations. Our algorithms leverage a new combination of ideas from the forward-reflected-backward splitting method…

In this paper we analyze a class of nonconvex optimization problem from the viewpoint of abstract convexity. Using the respective generalizations of the subgradient we propose an abstract notion proximal operator and derive a number of…

We consider compound geometric approximation for a nonnegative, integer-valued random variable $W$. The bound we give is straightforward but relies on having a lower bound on the failure rate of $W$. Applications are presented to M/G/1…

Our work focuses on stochastic gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer. Research on this class of problem is quite limited, and until recently no non-asymptotic convergence…

The theory of stochastic approximations form the theoretical foundation for studying convergence properties of many popular recursive learning algorithms in statistics, machine learning and statistical physics. Large deviations for…