Related papers: Smoothness of Subgradient Mappings and Its Applica…

Many machine learning problems involve regressing variables on a non-Euclidean manifold -- e.g. a discrete probability distribution, or the 6D pose of an object. One way to tackle these problems through gradient-based learning is to use a…

A new modulus of smoothness and its equivalent $K$-function are defined on the conic domains in $\mathbb{R}^d$, and used to characterize the weighted best approximation by polynomials. Both direct and weak inverse theorems of the…

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and…

In the setting of CAT(k) spaces, common fixed point iterations built from prox mappings (e.g. prox-prox, Krasnoselsky-Mann relaxations, nonlinear projected-gradients) converge locally linearly under the assumption of linear metric…

In view of training increasingly complex learning architectures, we establish a nonsmooth implicit function theorem with an operational calculus. Our result applies to most practical problems (i.e., definable problems) provided that a…

In this article we develop a general theory of exact parametric penalty functions for constrained optimization problems. The main advantage of the method of parametric penalty functions is the fact that a parametric penalty function can be…

As the problem of minimizing functionals on the Wasserstein space encompasses many applications in machine learning, different optimization algorithms on $\mathbb{R}^d$ have received their counterpart analog on the Wasserstein space. We…

Smoothness is crucial for attaining fast rates in first-order optimization. However, many optimization problems in modern machine learning involve non-smooth objectives. Recent studies relax the smoothness assumption by allowing the…

This paper is devoted to the generalized differential study of the normal cone mappings associated with a large class of parametric constraint systems (PCS) that appear, in particular, in nonpolyhedral conic programming. Conducting a local…

A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data. In this work, we show…

This paper introduces a stratification framework for nonlinear semidefinite programming (NLSDP) that reveals and utilizes the geometry behind the nonsmooth KKT system. Based on the \emph{index stratification} of $\mathbb{S}^n$ and its lift…

We show that the subgradient method converges only to local minimizers when applied to generic Lipschitz continuous and subdifferentially regular functions that are definable in an o-minimal structure. At a high level, the argument we…

In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of…

This paper considers non-smooth optimization problems where we seek to minimize the pointwise maximum of a continuously parameterized family of functions. Since the objective function is given as the solution to a maximization problem,…

There is a basic paradigm, called here the radius of well-posedness, which quantifies the "distance" from a given well-posed problem to the set of ill-posed problems of the same kind. In variational analysis, well-posedness is often…

This paper considers stochastic optimization problems for a large class of objective functions, including convex and continuous submodular. Stochastic proximal gradient methods have been widely used to solve such problems; however, their…

Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards models of low "complexity." We study the implicit…

Given a convex optimization problem and its dual, there are many possible first-order algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is…

We are concerned with a class of nonconvex and nonsmooth composite optimization problems, comprising a twice differentiable function and a prox-regular function. We establish a sufficient condition for the proximal mapping of a prox-regular…

This paper establishes the equivalence of the Aubin property and the strong regularity for generalized equations over $C^2$-cone reducible sets. This result resolves a long-standing question in variational analysis and extends the…