Related papers: How does the contraction property fail for convex …

Fitting a function by using linear combinations of a large number $N$ of `simple' components is one of the most fruitful ideas in statistical learning. This idea lies at the core of a variety of methods, from two-layer neural networks to…

We define a set inner product to be a function on pairs of convex bodies which is symmetric, Minkowski linear in each dimension, positive definite, and satisfies the natural analogue of the Cauchy-Schwartz inequality (which is not implied…

In these notes, we present a general result concerning the Lipschitz regularity of a certain type of set-valued maps often found in constrained optimization and control problems. The class of multifunctions examined in this paper is…

Error bounds are central objects in optimization theory and its applications. They were for a long time restricted only to the theory before becoming over the course of time a field of itself. This paper is devoted to the study of error…

Large learning rates, when applied to gradient descent for nonconvex optimization, yield various implicit biases including the edge of stability (Cohen et al., 2021), balancing (Wang et al., 2022), and catapult (Lewkowycz et al., 2020).…

Let $N$ be a weakly unitarily invariant norm (i.e. invariant for the coadjoint action of the unitary group) in the space of skew-Hermitian matrices $\mathfrak{u}_n(\mathbb C)$. In this paper we study the geometry of the unit sphere of such…

We study non-convex subgradient flows for training two-layer ReLU neural networks from a convex geometry and duality perspective. We characterize the implicit bias of unregularized non-convex gradient flow as convex regularization of an…

In this paper, we introduce the notion of strongly {\varphi}-convex functions with respect to c>0 and present some properties and representation of such functions. We obtain a characterization of inner product spaces involving the notion of…

Gradient descent-ascent (GDA) flows play a central role in finding saddle points of bivariate functionals, with applications in optimization, game theory, and robust control. While they are well-understood in Hilbert and Banach spaces via…

We introduce a family of functionals on submanifolds of Cartan-Hadamard manifolds that generalize the Colding-Minicozzi entropy of submanifolds of Euclidean space. We show that these functionals are monotone under mean curvature flow under…

We introduce both the notions of tensor product of convex bodies that contain zero in the interior, and of tensor product of $0$-symmetric convex bodies in Euclidean spaces. We prove that there is a bijection between tensor products of…

Non-convex functions that yet satisfy a condition of uniform convexity for non-close points can arise in discrete constructions. We prove that this sort of discrete uniform convexity is inherited by the convex envelope, which is the key to…

Orthogonally invariant functions of symmetric matrices often inherit properties from their diagonal restrictions: von Neumann's theorem on matrix norms is an early example. We discuss the example of "identifiability", a common property of…

Invariance (defined in a general sense) has been one of the most effective priors for representation learning. Direct factorization of parametric models is feasible only for a small range of invariances, while regularization approaches,…

In this paper, we consider optimization problems over closed embedded submanifolds of $\mathbb{R}^n$, which are defined by the constraints $c(x) = 0$. We propose a class of constraint dissolving approaches for these Riemannian optimization…

We consider optimization problems on manifolds with equality and inequality constraints. A large body of work treats constrained optimization in Euclidean spaces. In this work, we consider extensions of existing algorithms from the…

We consider the space of convex functions defined in the Euclidean $n$-dimensional space, which are lower semi-continuous and tend to infinity at infinity. We study real-valued valuations defined on this space of functions, which are…

A new notion of displacement convexity on a matrix level is developed for density flows arising from mean-field games, compressible Euler equations, entropic interpolation, and semi-classical limits of non-linear Schr\"odinger equations.…

This paper deals with nonsmooth convex optimization problems in Euclidean spaces. We identify special elements of the subdifferential of a convex function, called specular gradients. Based on this observation, we propose three numerical…

The g-convexity of functions on manifolds is a generalization of the convexity of functions on Rn. It plays an essential role in both differential geometry and non-convex optimization theory. This paper is concerned with g-convex smooth…