Related papers: Primal-Dual Stability in Local Optimality
In this work, we shall study the nonlinear inverse problems of recovering the Robin coefficients in elliptic and parabolic systems of second order, and establish their local Lipschitz stabilities. Some local Lipschitz stability was derived…
We study local controllability and optimal control problems for invertible discrete-time control systems. We present second order necessary conditions for optimality and sufficient conditions for local controllability. The conditions are…
We reconsider the stochastic (sub)gradient approach to the unconstrained primal L1-SVM optimization. We observe that if the learning rate is inversely proportional to the number of steps, i.e., the number of times any training pattern is…
We consider empirical risk minimization of linear predictors with convex loss functions. Such problems can be reformulated as convex-concave saddle point problems, and thus are well suitable for primal-dual first-order algorithms. However,…
This paper shows that the implicit bias of gradient descent on linearly separable data is exactly characterized by the optimal solution of a dual optimization problem given by a smoothed margin, even for general losses. This is in contrast…
Over the last years, minimization problems over spaces of measures have received increased interest due to their relevance in the context of inverse problems, optimal control and machine learning. A fundamental role in their numerical…
We consider the problem of analyzing and designing gradient-based discrete-time optimization algorithms for a class of unconstrained optimization problems having strongly convex objective functions with Lipschitz continuous gradient. By…
Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the…
We study a class of optimal control problems governed by nonlinear stochastic equations of monotone type under certain coercivity and linear growth conditions. We give first order necessary conditions of optimality. A stochastic Pontryagin…
For a second-order elliptic equation of nondivergence form in the plane, we investigate conditions on the coefficients which imply that all strong solutions have first-order derivatives that are Lipschitz continuous or differentiable at a…
Gradient Descent (GD) is a powerful workhorse of modern machine learning thanks to its scalability and efficiency in high-dimensional spaces. Its ability to find local minimisers is only guaranteed for losses with Lipschitz gradients, where…
Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in…
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing…
We present a general method for obtaining strong bounds for discrete optimization problems that is based on a concept of branching duality. It can be applied when no useful integer programming model is available, and we illustrate this with…
Binary optimization, a representative subclass of discrete optimization, plays an important role in mathematical optimization and has various applications in computer vision and machine learning. Usually, binary optimization problems are…
We study unconstrained Online Linear Optimization with Lipschitz losses. Motivated by the pursuit of instance optimality, we propose a new algorithm that simultaneously achieves ($i$) the AdaGrad-style second order gradient adaptivity; and…
Classical primal-dual affine programming takes place over finite dimensional real vector spaces. This results in beautiful duality theory, connecting the optimal solu- tions of the primal maximization problem and the dual minimization…
This paper focuses on investigating an inexact stochastic model-based optimization algorithm that integrates preconditioning techniques for solving stochastic composite optimization problems. The proposed framework unifies and extends the…
We study the problem of super-resolution, where we recover the locations and weights of non-negative point sources from a few samples of their convolution with a Gaussian kernel. It has been recently shown that exact recovery is possible by…
The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to…