Related papers: Nonlinear Randomized Urn Models: a Stochastic Appr…
This paper presents the link between stochastic approximation and clinical trials based on randomized urn models investigated in Bai and Hu (1999,2005) and Bai, Hu and Shen (2002). We reformulate the dynamics of both the urn composition and…
Stochastic approximation algorithms have been the subject of an enormous body of literature, both theoretical and applied. Recently, Laruelle and Pag\`es (2013) presented a link between the stochastic approximation and response-adaptive…
Stochastic Approximation (SA) is a popular approach for solving fixed-point equations where the information is corrupted by noise. In this paper, we consider an SA involving a contraction mapping with respect to an arbitrary norm, and show…
Stochastic approximation algorithm is a useful technique which has been exploited successfully in probability theory and statistics for a long time. The step sizes used in stochastic approximation are generally taken to be deterministic and…
Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous studies have demonstrated that the convergence rate of SA can be improved by introducing either smoothness or strong convexity condition. In…
We consider an urn model with multiple drawing and random time-dependent addition matrix. The model is very general with respect to previous literature: the number of sampled balls at each time-step is random, the addition matrix has…
We revisit the sample average approximation (SAA) approach for non-convex stochastic programming. We show that applying the SAA approach to problems with expected value equality constraints does not necessarily result in asymptotic…
This paper develops an analytic theory for the study of some Polya urns with random rules. The idea is to extend the isomorphism theorem in Flajolet et al. (2006), which connects deterministic balanced urns to a differential system for the…
Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with…
This paper introduces and analyzes a particular class of Polya urns: balls are of two colors, can only be added (the urns are said to be additive) and at every step the same constant number of balls is added, thus only the color…
This paper studies the control-oriented identification problem of set-valued moving average systems with uniform persistent excitations and observation noises. A stochastic approximation-based (SA-based) algorithm without projections or…
Stochastic approximation (SA) is a key method used in statistical learning. Recently, its non-asymptotic convergence analysis has been considered in many papers. However, most of the prior analyses are made under restrictive assumptions…
Consider a generalized time-dependent P\'olya urn process defined as follows. Let $d\in \mathbb{N}$ be the number of urns/colors. At each time $n$, we distribute $\sigma_n$ balls randomly to the $d$ urns, proportionally to $f$, where $f$ is…
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show…
This paper proposes a new family of algorithms for training neural networks (NNs). These are based on recent developments in the field of non-convex optimization, going under the general name of successive convex approximation (SCA)…
This paper considers the problem of minimizing an expectation function over a closed convex set, coupled with a {\color{black} functional or expectation} constraint on either decision variables or problem parameters. We first present a new…
This paper investigates the stability and convergence properties of asynchronous stochastic approximation (SA) algorithms, with a focus on extensions relevant to average-reward reinforcement learning. We first extend a stability proof…
We collect, survey and develop methods of (one-dimensional) stochastic approximation in a framework that seems suitable to handle fairly broad generalizations of Polya urns. To show the applicability of the results we determine the limiting…
We propose dynamic sampled stochastic approximation (SA) methods for stochastic optimization with a heavy-tailed distribution (with finite 2nd moment). The objective is the sum of a smooth convex function with a convex regularizer.…
A classical P\'olya urn scheme is a Markov process whose evolution is encoded by a replacement matrix $(R_{i,j})_{1\leq i,j\leq d}$. At every discrete time-step, we draw a ball uniformly at random, denote its colour $c$, and replace it in…