Related papers: Lower Envelopes and Steepest Descent Directions in…
Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gradient flow is amenable to theoretical analysis, but is…
This paper presents an algorithmic framework for the minimization of strictly convex quadratic functions. The framework is flexible and generic. At every iteration the search direction is a linear combination of the negative gradient, as…
This paper introduces and studies the convergence properties of a new class of explicit $\epsilon$-subgradient methods for the task of minimizing a convex function over the set of minimizers of another convex minimization problem. The…
Constrained non-convex optimization is fundamentally challenging, as global solutions are generally intractable and constraint qualifications may not hold. However, in many applications, including safe policy optimization in control and…
In this paper we study the radial epiderivative notion for nonconvex functions, which extends the (classical) directional derivative concept. The paper presents new definition and new properties for this notion and establishes relationships…
We study when the \emph{optimization curve} of first-order methods -- the sequence \${f(x\_n)}*{n\ge0}\$ produced by constant-stepsize iterations -- is convex, equivalently when the forward differences \$f(x\_n)-f(x*{n+1})\$ are…
Set- and vector-valued optimization problems can be re-formulated as complete lattice-valued problems. This has several advantages, one of which is the existence of a clear-cut solution concept which includes the attainment as the infimum…
In this work, the notions of normal cones at infinity to unbounded sets and limiting and singular subdifferentials at infinity for extended real value functions are introduced. Various calculus rules for these notions objects are…
In robust optimization, the general aim is to find a solution that performs well over a set of possible parameter outcomes, the so-called uncertainty set. In this paper, we assume that the uncertainty size is not fixed, and instead aim at…
The main purpose of this note is to establish the continuity of seminorms on finite-dimensional vector spaces over the real or complex numbers.
The paper is devoted to deriving necessary optimality conditions in a general optimal control problem for dynamical systems governed by controlled sweeping processes with hard-constrained control actions entering both polyhedral moving sets…
We introduce a new class of inverse optimization problems in which an input solution is given together with $k$ linear weight functions, and the goal is to modify the weights by the same deviation vector $p$ so that the input solution…
We introduce and study a generalized concept of boundedness of a subset of a normed vector space with respect to a cone, which is defined as lower boundedness of the images of the underlying set through all the positive functionals of the…
This paper presents a novel convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems with non-convex constraints that restrict the…
Inspired by regularization techniques in statistics and machine learning, we study complementary composite minimization in the stochastic setting. This problem corresponds to the minimization of the sum of a (weakly) smooth function endowed…
This work extends our previous study from S. Shrestha et al. (2024) by introducing a new abstract framework for Variational Multiscale (VMS) methods at the discrete level. We introduce the concept of what we define as the optimal projector…
The discrete moment problem is a foundational problem in distribution-free robust optimization, where the goal is to find a worst-case distribution that satisfies a given set of moments. This paper studies the discrete moment problems with…
This paper studies the problem of optimal switching for one-dimensional diffusion, which may be regarded as sequential optimal stopping problem with changes of regimes. The resulting dynamic programming principle leads to a system of…
We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and…
At each iteration of a Block Coordinate Descent method one minimizes an approximation of the objective function with respect to a generally small set of variables subject to constraints in which these variables are involved. The…