Related papers: Second-Order Newton-Based Extremum Seeking for Mul…
We study the sparse phase retrieval problem, which seeks to recover a sparse signal from a limited set of magnitude-only measurements. In contrast to prevalent sparse phase retrieval algorithms that primarily use first-order methods, we…
We design an algorithm which finds an $\epsilon$-approximate stationary point (with $\|\nabla F(x)\|\le \epsilon$) using $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector products, matching guarantees that were previously available…
We propose Sketched Online Newton (SON), an online second order learning algorithm that enjoys substantially improved regret guarantees for ill-conditioned data. SON is an enhanced version of the Online Newton Step, which, via sketching…
This work proposes an accelerated first-order algorithm we call the Robust Momentum Method for optimizing smooth strongly convex functions. The algorithm has a single scalar parameter that can be tuned to trade off robustness to gradient…
This paper focuses on systems of nonlinear second-order stochastic differential equations with multi-scales. The motivation for our study stems from mathematical physics and statistical mechanics, for examples, Langevin dynamics and…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient…
Second-order optimization methods have desirable convergence properties. However, the exact Newton method requires expensive computation for the Hessian and its inverse. In this paper, we propose SPAN, a novel approximate and fast Newton…
We present results on stabilization for reduced order models (ROM) of partial differential equations using learning. Stabilization is achieved via closure models for ROMs, where we use a model-free extremum seeking (ES) dither-based…
In this paper we derive new second-order optimality conditions for a very general set-constrained optimization problem where the underlying set may be nononvex. We consider local optimality in specific directions (i.e., optimal in a…
We present an algorithm for steering the output of a linear system from a feasible initial condition to a desired target position, while satisfying input constraints and non-convex output constraints. The system input is generated by a…
This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding $x^\star \in \mathcal{X}$ such that…
The following document presents some novel numerical methods valid for one and several variables, which using the fractional derivative, allow to find solutions for some non-linear systems in the complex space using real initial conditions.…
The method of location and spectral estimation of weak signals on a noise background is being considered. The method is based on the optimized on order and noise dispersion autoregressive model of a sought signal. A new approach of model…
We present some results on stabilization for reduced-order models (ROMs) of partial differential equations. The stabilization is achieved using Lyapunov theory to design a new closure model that is robust to parametric uncertainties. The…
This paper studies distributed estimation and inference for a general statistical problem with a convex loss that could be non-differentiable. For the purpose of efficient computation, we restrict ourselves to stochastic first-order…
The motivation of this work is an inverse problem for the acoustic wave equation, where an array of sensors probes an unknown medium with pulses and measures the scattered waves. The goal of the inversion is to determine from these…
We propose a novel optimization algorithm for continuous functions using geodesics and contours under conformal mapping.The algorithm can find multiple optima by first following a geodesic curve to a local optimum then traveling to the next…
This paper addresses second-order stochastic optimization for estimating the minimizer of a convex function written as an expectation. A direct recursive estimation technique for the inverse Hessian matrix using a Robbins-Monro procedure is…
Active perception is a fundamental problem in autonomous robotics in which the robot must decide where to move and what to sense in order to obtain the most informative observations for accomplishing its mission. Existing approaches either…