Related papers: Non-local Optimization: Imposing Structure on Opti…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…
Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations.…
We study the problem of global maximization of a function f given a finite number of evaluations perturbed by noise. We consider a very weak assumption on the function, namely that it is locally smooth (in some precise sense) with respect…
We present a general numerical approach to shape optimization with state constraints for 2-dimensional geometries, without relaxing the constraints. To do this we reformulate the problem on a fixed reference domain using a conformal…
This paper describes a node relocation algorithm based on nonlinear optimization which delivers excellent results for both unstructured and structured plane triangle meshes over convex as well as non-convex domains with high curvature. The…
Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and statistics. Although deterministic algorithms for DC…
In non-private stochastic convex optimization, stochastic gradient methods converge much faster on interpolation problems -- problems where there exists a solution that simultaneously minimizes all of the sample losses -- than on…
Over the past ten years, driven by large scale optimisation problems arising from machine learning, the development of stochastic optimisation methods have witnessed a tremendous growth. However, despite their popularity, the theoretical…
One of the most effective algorithms for differentially private learning and optimization is objective perturbation. This technique augments a given optimization problem (e.g. deriving from an ERM problem) with a random linear term, and…
We study contextual stochastic optimization problems, where we leverage rich auxiliary observations (e.g., product characteristics) to improve decision making with uncertain variables (e.g., demand). We show how to train forest decision…
We consider the problem of minimizing a convex objective function $F$ when one can only evaluate its noisy approximation $\hat{F}$. Unless one assumes some structure on the noise, $\hat{F}$ may be an arbitrary nonconvex function, making the…
We consider minimization of composite functions of the form $f(g(x))+h(x)$, where $f$ and $h$ are convex functions (which can be nonsmooth) and $g$ is a smooth vector mapping. In addition, we assume that $g$ is the average of finite number…
Gradient clipping is a commonly used technique to stabilize the training process of neural networks. A growing body of studies has shown that gradient clipping is a promising technique for dealing with the heavy-tailed behavior that emerged…
This work addresses the occupation measure relaxation of calculus of variations problems, which is an infinite-dimensional linear programming relaxation amenable to numerical approximation by a hierarchy of semidefinite optimization…
This paper deals with shape optimization for elastic materials under stochastic loads. It transfers the paradigm of stochastic dominance, which allows for flexible risk aversion via comparison with benchmark random variables, from…
We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error. Our approach…
We propose a new gradient descent algorithm with added stochastic terms for finding the global optimizers of nonconvex optimization problems. A key component in the algorithm is the adaptive tuning of the randomness based on the value of…
In this research, we investigate a general shape optimization problem in which the state equation is expressed using a nonlocal and nonlinear operator. We prove the existence of a minimum point for a functional $F$ defined on the family of…
Classical worst-case optimization theory neither explains the success of optimization in machine learning, nor does it help with step size selection. In this paper we demonstrate the viability and advantages of replacing the classical…
We provide theory for computing the lower semi-continuous convex envelope of functionals of the type f(x) plus an l2 misfit, and discuss applications to various non-convex optimization problems. The latter term is a data fit term whereas f…