Related papers: Generic nondegeneracy in convex optimization

While the optimization problem behind deep neural networks is highly non-convex, it is frequently observed in practice that training deep networks seems possible without getting stuck in suboptimal points. It has been argued that this is…

We study $H^1$ versus $C^1$ local minimizers for functionals defined on spaces of symmetric functions, namely functions that are invariant by the action of some subgroups of $\mathcal{O}(N)$. These functionals, in many cases, are associated…

Nonconvex optimization problems arise in many areas of computational science and engineering and are (approximately) solved by a variety of algorithms. Existing algorithms usually only have local convergence or subsequence convergence of…

For a large class of variational problems we prove that minimizers are symmetric whenever they are $C^1$.

This paper presents an algorithmic framework for the minimization of strictly convex quadratic functions. The framework is flexible and generic. At every iteration the search direction is a linear combination of the negative gradient, as…

We discuss non-Euclidean deterministic and stochastic algorithms for optimization problems with strongly and uniformly convex objectives. We provide accuracy bounds for the performance of these algorithms and design methods which are…

We study operator log-convex functions on $(0,\infty)$, and prove that a continuous nonnegative function on $(0,\infty)$ is operator log-convex if and only if it is operator monotone decreasing. Several equivalent conditions related to…

Techniques involving factorization are found in a wide range of applications and have enjoyed significant empirical success in many fields. However, common to a vast majority of these problems is the significant disadvantage that the…

We propose convex relaxations for convolutional neural nets with one hidden layer where the output weights are fixed. For convex activation functions such as rectified linear units, the relaxations are convex second order cone programs…

The main result of the paper is on the continuity of weak solutions of infinitely degenerate quasilinear second order equations. Namely, we show that every weak solution to a certain class of degenerate quasilinear equations is continuous.…

A framework is presented whereby a general convex conic optimization problem is transformed into an equivalent convex optimization problem whose only constraints are linear equations and whose objective function is Lipschitz continuous.…

In this work, we show the consistency of an approach for solving robust optimization problems using sequences of sub-problems generated by ergodic measure preserving transformations. The main result of this paper is that the minimizers and…

In this paper we present a general convex optimization approach for solving high-dimensional multiple response tensor regression problems under low-dimensional structural assumptions. We consider using convex and weakly decomposable…

Composite minimization involves a collection of functions which are aggregated in a nonsmooth manner. It covers, as a particular case, smooth approximation of minimax games, minimization of max-type functions, and simple composite…

Mixed functions are analytic functions in variables $z_1,..., z_n$ and their conjugates $\bar z_1,..., \bar z_n$. We introduce the notion of Newton non-degeneracy for mixed functions and develop a basic tool for the study of mixed…

We prove convergence of a single time-scale stochastic subgradient method with subgradient averaging for constrained problems with a nonsmooth and nonconvex objective function having the property of generalized differentiability. As a tool…

Motivated by recent increased interest in optimization algorithms for non-convex optimization in application to training deep neural networks and other optimization problems in data analysis, we give an overview of recent theoretical…

Many problems of theoretical and practical interest involve finding a convex or concave function. For instance, optimization problems such as finding the projection on the convex functions in $H^k(\Omega)$, or some problems in economics. In…

The aim of this article is to establish new two-functions minimax inequalities extending classical results such as Simons' minimax theorem. Our results will be proved in a non-compact setting. We also prove, under general conditions, that…

This paper presents an algorithm for approximately minimizing a convex function in simple, not necessarily bounded convex domains, assuming only that function values and subgradients are available. No global information about the objective…