Related papers: Understanding Notions of Stationarity in Non-Smoot…
Large-scale nonconvex optimization problems are ubiquitous in modern machine learning, and among practitioners interested in solving them, Stochastic Gradient Descent (SGD) reigns supreme. We revisit the analysis of SGD in the nonconvex…
Many specific problems ranging from theoretical probability to applications in statistical physics, combinatorial optimization and communications can be formulated as an optimal tuning of local parameters in large systems of interacting…
We study the smoothness of the stationary measure with respect to smooth perturbations of the iterated function scheme and the weight functions that define it. Our main theorems relate the smoothness of the perturbation of: the iterated…
In this work we investigate stochastic non-convex optimization problems where the objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such…
Over the past decades, the concept "partial smoothness" has been playing as a powerful tool in several fields involving nonsmooth analysis, such as nonsmooth optimization, inverse problems and operation research, etc. The essence of partial…
In Part I of this paper, we proposed and analyzed a novel algorithmic framework for the minimization of a nonconvex (smooth) objective function, subject to nonconvex constraints, based on inner convex approximations. This Part II is devoted…
In this article we utilise abstract convexity theory in order to unify and generalize many different concepts from nonsmooth analysis. We introduce the concepts of abstract codifferentiability, abstract quasidifferentiability and abstract…
Minimizing a smooth function f on a closed subset C leads to different notions of stationarity: Fr{\'e}chet stationarity, which carries a strong variational meaning, and criticality, which is defined through a closure process and involves…
In this paper we consider a sufficiently broad class of nonlinear mathematical programs with disjunctive constraints, which, e.g., include mathematical programs with complemetarity/vanishing constraints. We present an extension of the…
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…
We consider a class of Riemannian optimization problems where the objective is the sum of a smooth function and a nonsmooth function, considered in the ambient space. This class of problems finds important applications in machine learning…
Approximations of optimization problems arise in computational procedures and sensitivity analysis. The resulting effect on solutions can be significant, with even small approximations of components of a problem translating into large…
We analyze convergence rates of stochastic optimization procedures for non-smooth convex optimization problems. By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates of stochastic…
Assessment of the degree of boundedness/stability of multidimensional nonlinear systems with time-dependent and nonperiodic coefficients is an important problem in various applied areas which has no adequate resolution yet. Most of the…
We investigate the local topological structure, stationary point sets in parametric optimization genericly may have. Our main result states that, up to stratified isomorphism, any such structure is already present in the small subclass of…
Sparse learning is a very important tool for mining useful information and patterns from high dimensional data. Non-convex non-smooth regularized learning problems play essential roles in sparse learning, and have drawn extensive attentions…
Many spatial processes exhibit nonstationary features. We estimate a variance function from a single process observation where the errors are nonstationary and correlated. We propose a difference-based approach for a one-dimensional…
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and…
This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets,…
Graph-based methods for signal processing have shown promise for the analysis of data exhibiting irregular structure, such as those found in social, transportation, and sensor networks. Yet, though these systems are often dynamic,…