Related papers: Variational Optimization on Lie Groups, with Examp…
We provide several quantum algorithms for continuous optimization that do not require gradient estimation. Instead, we encode the optimization problem into the dynamics of a physical system and coherently simulate the time evolution. We…
In this letter, we study distributed optimization and Nash equilibrium-seeking dynamics from a contraction theoretic perspective. Our first result is a novel bound on the logarithmic norm of saddle matrices. Second, for distributed gradient…
Composite optimization problems, where a smooth loss is combined with a nonsmooth regularizer, are common in machine learning and inverse problems. In this work, we study a proximal extension of NAG-GS, a semi-implicit accelerated method…
In this paper, we propose objective-function-free (OFF) variants of the proximal Newton method for nonconvex composite optimization problems and the regularized Newton method for unconstrained optimization problems, respectively, using…
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing…
We introduce adaptive, tuning-free step size schedules for gradient-based sampling algorithms obtained as time-discretizations of Wasserstein gradient flows. The result is a suite of tuning-free sampling algorithms, including tuning-free…
Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime. Much work has focused on the case where the…
Left-invariant optimal control problems on Lie groups form an important class of problems with big symmetry group. They are interesting from the theoretical point of view since they often can be completely studied, and general features can…
Stochastic bilevel optimization (SBO) has been integrated into many machine learning paradigms recently, including hyperparameter optimization, meta learning, and reinforcement learning. Along with the wide range of applications, there have…
Distributed aggregative optimization underpins many cooperative optimization and multi-agent control systems, where each agent's objective function depends both on its local optimization variable and an aggregate of all agents' optimization…
Several problems in number theory when reformulated in terms of homogenous dynamics involve study of limiting distributions of translates of algebraically defined measures on orbits of reductive groups. The general non-divergence and…
We propose a new class of finite element approximations to ideal compressible magnetohydrodynamic equations in smooth regime. Following variational approximations developed for fluid models in the last decade, our discretizations are built…
The maximum (or minimum) generalized eigenvalue of symmetric positive semidefinite matrices that depend on optimization variables often appears as objective or constraint functions in structural topology optimization when we consider…
Variational analysis provides the theoretical foundations and practical tools for constructing optimization algorithms without being restricted to smooth or convex problems. We survey the central concepts in the context of a concrete but…
We consider a continuous-time optimization method based on a dynamical system, where a massive particle starting at rest moves in the conservative force field generated by the objective function, without any kind of friction. We formulate a…
Lie scale invariance is used to reduce the incompressible Navier-Stokes equations to non-linear ordinary equations. This yields a formulation in terms of logarithmic spirals as independent variables. We give the equations when the spirals…
We propose a Distributionally Robust Optimization (DRO) formulation with a Wasserstein-based uncertainty set for selecting grouped variables under perturbations on the data for both linear regression and classification problems. The…
Large-scale constrained optimization problems are at the core of many tasks in control, signal processing, and machine learning. Notably, problems with functional constraints arise when, beyond a performance{\nobreakdash-}centric goal…
We introduce a new multimodal optimization approach called Natural Variational Annealing (NVA) that combines the strengths of three foundational concepts to simultaneously search for multiple global and local modes of black-box nonconvex…
The stochastic gradient Langevin Dynamics is one of the most fundamental algorithms to solve sampling problems and non-convex optimization appearing in several machine learning applications. Especially, its variance reduced versions have…