Related papers: Online Inference of Constrained Optimization: Prim…

Stochastic gradient methods (SGMs) have been widely used for solving stochastic optimization problems. A majority of existing works assume no constraints or easy-to-project constraints. In this paper, we consider convex stochastic…

Stochastic gradient descent (SGD) and projected stochastic gradient descent (PSGD) are scalable algorithms to compute model parameters in unconstrained and constrained optimization problems. In comparison with SGD, PSGD forces its iterative…

We introduce an algorithm called SQDP (Stochastic Quadratic Dynamic Programming) to solve some multistage stochastic optimization problems having strongly convex recourse functions. The algorithm extends the classical Stochastic Dual…

The nonlinear optimization problem with linear constraints has many applications in engineering fields such as the visual-inertial navigation and localization of an unmanned aerial vehicle maintaining the horizontal flight. In order to…

The stochastic subgradient method is a widely-used algorithm for solving large-scale optimization problems arising in machine learning. Often these problems are neither smooth nor convex. Recently, Davis et al. [1-2] characterized the…

In this paper, we develop a unified framework for analyzing the tracking error and dynamic regret of inexact online optimization methods under a variety of settings. Specifically, we leverage the quadratic constraint approach from control…

Stochastic Optimization is a cornerstone of operations research, providing a framework to solve optimization problems under uncertainty. Despite the development of numerous algorithms to tackle these problems, several persistent challenges…

We analyze a sequential quadratic programming algorithm for solving a class of abstract optimization problems. Assuming that the initial point is in an $L^2$ neighborhood of a local solution that satisfies no-gap second-order sufficient…

Convex quadratic programs (QPs) constitute a fundamental computational primitive across diverse domains including financial optimization, control systems, and machine learning. The alternating direction method of multipliers (ADMM) has…

Motivated by applications in optimization and machine learning, we consider stochastic quasi-Newton (SQN) methods for solving stochastic optimization problems. In the literature, the convergence analysis of these algorithms relies on strong…

This paper explores numerical methods for solving a convex differentiable semi-infinite program. We introduce a primal-dual gradient method which performs three updates iteratively: a momentum gradient ascend step to update the constraint…

In this paper we propose a stochastic primal dual fixed point method (SPDFP) for solving the sum of two proper lower semi-continuous convex function and one of which is composite. The method is based on the primal dual fixed point method…

In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadratic Programming (TR-SSQP) method and establish its…

Partition functions arise in a variety of settings, including conditional random fields, logistic regression, and latent gaussian models. In this paper, we consider semistochastic quadratic bound (SQB) methods for maximum likelihood…

We propose a novel algorithm, TR-SVR, for solving unconstrained stochastic optimization problems. This method builds on the trust-region framework, which effectively balances local and global exploration in optimization tasks. TR-SVR…

We propose a novel stochastic approximation algorithm, termed PMQSopt, for solving weakly convex stochastic optimization problems involving expectation-valued functions. The algorithm is constructed by integrating the proximal method of…

We prove that a "first-order" Sequential Quadratic Programming (SQP) algorithm for equality constrained optimization has local linear convergence with rate $(1-1/\kappa_R)^k$, where $\kappa_R$ is the condition number of the Riemannian…

Motivated by the problem of online canonical correlation analysis, we propose the \emph{Stochastic Scaled-Gradient Descent} (SSGD) algorithm for minimizing the expectation of a stochastic function over a generic Riemannian manifold. SSGD…

Attention to data-driven optimization approaches, including the well-known stochastic gradient descent method, has grown significantly over recent decades, but data-driven constraints have rarely been studied, because of the computational…

The stochastic gradient descent (SGD) algorithm is widely used for parameter estimation, especially for huge data sets and online learning. While this recursive algorithm is popular for computation and memory efficiency, quantifying…