Related papers: Nonlinearly preconditioned gradient flows
In this paper, we derive a number of interesting properties and extensions of the convex flow problem from the perspective of convex geometry. We show that the sets of allowable flows always can be imbued with a downward closure property,…
Stochastic gradient methods with momentum are widely used in applications and at the core of optimization subroutines in many popular machine learning libraries. However, their sample complexities have not been obtained for problems beyond…
We study the long-time behavior of global strong solutions to a hydrodynamic system for nonhomogeneous incompressible nematic liquid crystal flows driven by two types of external forces in a smooth bounded domain in $\mathbb{R}^2$. For…
We investigate the convergence of an application of a proximal gradient method to control problems with nonsmooth and nonconvex control cost. Here, we focus on control cost functionals that promote sparsity, which includes functionals of…
Recent work by Woodworth et al. (2020) shows that the optimization dynamics of gradient descent for overparameterized problems can be viewed as low-dimensional dual dynamics induced by a mirror map, explaining the implicit regularization…
With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical…
We consider stochastic optimization problems involving an expected value of a nonlinear function of a base random vector and a conditional expectation of another function depending on the base random vector, a dependent random vector, and…
In the paper, we propose a class of efficient adaptive bilevel methods based on mirror descent for nonconvex bilevel optimization, where its upper-level problem is nonconvex possibly with nonsmooth regularization, and its lower-level…
We propose a single time-scale stochastic subgradient method for constrained optimization of a composition of several nonsmooth and nonconvex functions. The functions are assumed to be locally Lipschitz and differentiable in a generalized…
This brief gives a set of unified Lyapunov stability conditions to guarantee the predefined-time/finite-time stability of a dynamical systems. The derived Lyapunov theorem for autonomous systems establishes equivalence with existing…
We develop structure preserving schemes for a class of nonlinear mobility continuity equation. When the mobility is a concave function, this equation admits a form of gradient flow with respect to a Wasserstein-like transport metric. Our…
We provide several characterizations of convergence to unstable equilibria in nonlinear systems. Our current contribution is three-fold. First we present simple algebraic conditions for establishing local convergence of non-trivial…
We consider stochastic gradient methods under the interpolation regime where a perfect fit can be obtained (minimum loss at each observation). While previous work highlighted the implicit regularization of such algorithms, we consider an…
Continuous time primal-dual gradient dynamics that find a saddle point of a Lagrangian of an optimization problem have been widely used in systems and control. While the global asymptotic stability of such dynamics has been well-studied, it…
We consider the evolution of open planar curves by the steepest descent flow of a geometric functional, under different boundary conditions. We prove that, if any set of stationary solutions with fixed energy is finite, then a solution of…
Neural networks trained to minimize the logistic (a.k.a. cross-entropy) loss with gradient-based methods are observed to perform well in many supervised classification tasks. Towards understanding this phenomenon, we analyze the training…
In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problems. We provide a short proof that establishes the linear…
This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local…
Following Demidovich's concept and definition of convergent systems, we analyze the optimal nonlinear damping control, recently proposed [1] for the second-order systems. Targeting the problem of output regulation, correspondingly tracking…
We prove convergence of the proximal policy gradient method for a class of constrained stochastic control problems with control in both the drift and diffusion of the state process. The problem requires either the running or terminal cost…