Related papers: On Certain Conditions for Convex Optimization in H…
This paper investigates solution strategies for nonlinear problems in Hilbert spaces, such as nonlinear partial differential equations (PDEs) in Sobolev spaces, when only finite measurements are available. We formulate this as a nonlinear…
We consider gradient flow/gradient descent and heavy ball/accelerated gradient descent optimization for convex objective functions. In the gradient flow case, we prove the following: 1. If $f$ does not have a minimizer, the convergence…
In this paper, in terms of three types of generalized second-order derivatives of a nonsmooth function, we mainly study the corresponding second-order optimality conditions in a Hilbert space and prove the equivalence among these optimality…
Minimax optimization has been central in addressing various applications in machine learning, game theory, and control theory. Prior literature has thus far mainly focused on studying such problems in the continuous domain, e.g.,…
In this paper, we provide conditions under which one can take derivatives of the solution to convex optimization problems with respect to problem data. These conditions are (roughly) that Slater's condition holds, the functions involved are…
Non-convex optimization is ubiquitous in modern machine learning. Researchers devise non-convex objective functions and optimize them using off-the-shelf optimizers such as stochastic gradient descent and its variants, which leverage the…
We give an iteration scheme for finding zeros of maximal monotone operators in Hilbert spaces. We assume that the operator is defined in the whole space. The iterates converge strongly to a solution if there exists any, otherwise they tend…
In this paper, we propose second-order sufficient optimality conditions for a very general nonconvex constrained optimization problem, which covers many prominent mathematical programs.Unlike the existing results in the literature, our…
Many problems of theoretical and practical interest involve finding a convex or concave function. For instance, optimization problems such as finding the projection on the convex functions in $H^k(\Omega)$, or some problems in economics. In…
We present results about minimization of convex functionals defined over a finite set of vectors in a finite dimensional Hilbert space, that extend several known results for the Benedetto-Fickus frame potential. Our approach depends on…
In this paper we study convex bi-level optimization problems for which the inner level consists of minimization of the sum of smooth and nonsmooth functions. The outer level aims at minimizing a smooth and strongly convex function over the…
This paper discusses differential stability of convex programming problems in Hausdorff locally convex topological vector spaces. Among other things, we obtain formulas for computing or estimating the subdifferential and the singular…
In this paper, we study some optimization problems in uniformly convex and uniformly smooth Bochner spaces. We consider four cases of the underlying subsets: closed and convex subsets, closed and convex cones, closed subspaces and closed…
The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality…
Non-convex optimization problems can be approximately solved via relaxation or local algorithms. For many practical problems such as optimal power flow (OPF) problems, both approaches tend to succeed in the sense that relaxation is usually…
For finite-dimensional problems, stochastic approximation methods have long been used to solve stochastic optimization problems. Their application to infinite-dimensional problems is less understood, particularly for nonconvex objectives.…
In this paper we consider the computation of approximate solutions for inverse problems in Hilbert spaces. In order to capture the special feature of solutions, non-smooth convex functions are introduced as penalty terms. By exploiting the…
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non-convex constraint set introduced by model parameterization.…
Separable convex optimization problems with linear ascending inequality and equality constraints are addressed in this paper. Under an ordering condition on the slopes of the functions at the origin, an algorithm that determines the optimum…
This paper studies duality and optimality conditions for general convex stochastic optimization problems. The main result gives sufficient conditions for the absence of a duality gap and the existence of dual solutions in a locally convex…