Related papers: Finite-Time Analysis and Restarting Scheme for Lin…
We develop a Sequential Quadratic Optimization (SQP) algorithm for minimizing a stochastic objective function subject to deterministic equality constraints. The method utilizes two different stepsizes, one which exclusively scales the…
Restart is a general framework, of prime importance and wide applicability, for expediting first-passage times and completion times of general stochastic processes. Restart protocols can use either deterministic or stochastic timers.…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…
In energy management, it is common that strategic investment decisions (storage capacity, production units) are made at a slow time scale, whereas operational decisions (storage, production) are made at a fast time scale: for such problems,…
We study a variance reduction strategy based on control variables for simulating the averaged macroscopic behavior of a stochastic slow-fast system. We assume that this averaged behavior can be written in terms of a few slow degrees of…
A fully discrete approximation of the semi-linear stochastic wave equation driven by multiplicative noise is presented. A standard linear finite element approximation is used in space and a stochastic trigonometric method for the temporal…
In this work we propose a nonconvex two-stage \underline{s}tochastic \underline{a}lternating \underline{m}inimizing (SAM) method for sparse phase retrieval. The proposed algorithm is guaranteed to have an exact recovery from $O(s\log n)$…
This paper addresses identification of sparse linear and noise-driven continuous-time state-space systems, i.e., the right-hand sides in the dynamical equations depend only on a subset of the states. The key assumption in this study, is…
In this paper, we analyze the finite sample complexity of stochastic system identification using modern tools from machine learning and statistics. An unknown discrete-time linear system evolves over time under Gaussian noise without…
In the presence of multiscale dynamics in a reaction network, direct simulation methods become inefficient as they can only advance the system on the smallest scale. This work presents stochastic averaging techniques to accelerate…
We study a class of two-stage stochastic programs in which the second stage includes a set of components with uncertain capacity, and the expression for the distribution function of the uncertain capacity includes first-stage variables.…
Sharpness is an almost generic assumption in continuous optimization that bounds the distance from minima by objective function suboptimality. It facilitates the acceleration of first-order methods through restarts. However, sharpness…
Stochastic time-varying optimization is an integral part of learning in which the shape of the function changes over time in a non-deterministic manner. This paper considers multiple models of stochastic time variation and analyzes the…
Randomized iterative algorithms have attracted much attention in recent years because they can approximately solve large-scale linear systems of equations without accessing the entire coefficient matrix. In this paper, we propose two novel…
Theory and application of stochastic approximation (SA) have become increasingly relevant due in part to applications in optimization and reinforcement learning. This paper takes a new look at SA with constant step-size $\alpha>0$, defined…
In this paper, we study and analyze zeroth-order stochastic approximation algorithms for solving bilvel problems, when neither the upper/lower objective values, nor their unbiased gradient estimates are available. In particular, exploiting…
We consider jointly estimating the coefficient matrix and the error precision matrix in high-dimensional multivariate linear regression models. Bayesian methods in this context often face computational challenges, leading to previous…
We consider the problem of fitting variational posterior approximations using stochastic optimization methods. The performance of these approximations depends on (1) how well the variational family matches the true posterior…
This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization. Bilevel optimization is a class of problems which exhibit a two-level structure, and its goal is to minimize an outer objective function with…
We propose a novel time stepping method for linear poroelasticity by extending a recent iterative decoupling approach to the second-order case. This results in a two-step scheme with an inner iteration and a relaxation step. We prove…