Related papers: Decoupled Functional Central Limit Theorems for Tw…
We present adaptive sequential SAA (sample average approximation) algorithms to solve large-scale two-stage stochastic linear programs. The iterative algorithm framework we propose is organized into \emph{outer} and \emph{inner} iterations…
We discuss a few mathematical aspects of random dynamical decoupling, a key tool procedure in quantum information theory. In particular, we place it in the context of discrete stochastic processes, limit theorems and CPT semigroups on…
Consider generalized adapted stochastic integrals with respect to independently scattered random measures with second moments. We use a decoupling technique, known as the "principle of conditioning", to study their stable convergence…
In this paper, we establish non-asymptotic bounds for accuracy of normal approximation for linear two-timescale stochastic approximation (TTSA) algorithms driven by martingale difference or Markov noise. Focusing on both the last iterate…
This paper considers the problem of minimizing an expectation function over a closed convex set, coupled with a {\color{black} functional or expectation} constraint on either decision variables or problem parameters. We first present a new…
Motivated by the widespread use of temporal-difference (TD-) and Q-learning algorithms in reinforcement learning, this paper studies a class of biased stochastic approximation (SA) procedures under a mild "ergodic-like" assumption on the…
Policy evaluation in reinforcement learning is often conducted using two-timescale stochastic approximation, which results in various gradient temporal difference methods such as GTD(0), GTD2, and TDC. Here, we provide convergence rate…
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle…
Stochastic approximation (SA) that involves multiple coupled sequences, known as multiple-sequence SA (MSSA), finds diverse applications in the fields of signal processing and machine learning. However, existing theoretical understandings…
Two-time-scale stochastic approximation algorithms are iterative methods used in applications such as optimization, reinforcement learning, and control. Finite-time analysis of these algorithms has primarily focused on fixed point…
Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools and techniques that are popular in multi-agent and distributed control scenarios. To counter Bellman's curse of dimensionality, such…
This paper provides a Central Limit Theorem (CLT) for a process $\{\theta_n, n\geq 0\}$ satisfying a stochastic approximation (SA) equation of the form $\theta_{n+1} = \theta_n + \gamma_{n+1} H(\theta_n,X_{n+1})$; a CLT for the associated…
We obtain Central Limit Theorems in Functional form for a class of time-inhomogeneous interacting random walks on the simplex of probability measures over a finite set. Due to a reinforcement mechanism, the increments of the walks are…
Complex systems are often characterized by the interplay of multiple interconnected dynamical processes operating across a range of temporal scales. This phenomenon is widespread in both biological and artificial scenarios, making it…
We study asymptotic behaviour of stochastic approximation procedures with three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and a dynamically changing random regression function.…
We consider noisy non-synchronous discrete observations of a continuous semimartingale with random volatility. Functional stable central limit theorems are established under high-frequency asymptotics in three setups: one-dimensional for…
Stochastic approximation (SA) is a fundamental iterative framework with broad applications in reinforcement learning and optimization. Classical analyses typically rely on martingale difference or Markov noise with bounded second moments,…
We consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is…
Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it constant} stpesize schemes through the lens of Markov…
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show…