Related papers: Limit theorems for stochastic approximation algori…
In this paper we estimate the expected error of a stochastic approximation algorithm where the maximum of a function is found using finite differences of a stochastic representation of that function. An error estimate of $O(n^{-1/5})$ for…
We study stochastic convex optimization under infinite noise variance. Specifically, when the stochastic gradient is unbiased and has uniformly bounded $(1+\kappa)$-th moment, for some $\kappa \in (0,1]$, we quantify the convergence rate of…
We show through case studies that it is easier to estimate the fundamental limits of data processing than to construct explicit algorithms to achieve those limits. Focusing on binary classification, data compression, and prediction under…
We extend the methods and results of [arXiv 1603.04896] to the setting of multinomial distributions satisfying certain properties. These include all the multinomial distributions arising from the direct proof of the Central Limit Theorem…
Learning-to-optimize leverages machine learning to accelerate optimization algorithms. While empirical results show tremendous improvements compared to classical optimization algorithms, theoretical guarantees are mostly lacking, such that…
In this paper, we study a stochastic strongly convex optimization problem and propose three classes of variable sample-size stochastic first-order methods including the standard stochastic gradient descent method, its accelerated variant,…
This paper focuses on investigating an inexact stochastic model-based optimization algorithm that integrates preconditioning techniques for solving stochastic composite optimization problems. The proposed framework unifies and extends the…
The convergence rate of stochastic gradient search is analyzed in this paper. Using arguments based on differential geometry and Lojasiewicz inequalities, tight bounds on the convergence rate of general stochastic gradient algorithms are…
We give bounds for (central) moments for balanced P\'olya urns under very general conditions. In some cases, these bounds imply that moment convergence holds in earlier known results on asymptotic distribution. The results overlap with…
The global clustering coefficient serves as a powerful metric for the structural analysis and comparison of complex networks. Random geometric graphs offer a realistic framework for representing the spatial constraints and geometry often…
Adaptive and interacting Markov Chains Monte Carlo (MCMC) algorithms are a novel class of non-Markovian algorithms aimed at improving the simulation efficiency for complicated target distributions. In this paper, we study a general…
We explore the limit of stochastic differential equations driven by some random processes satisfying singularly perturbed second order stochastic differential equations. The main tool we employ is the universal limit theorem in rough path…
In this paper, we study the superconvergence phenomenon in the free central limit theorem for identically distributed, unbounded summands. We prove not only the uniform convergence of the densities to the semicircular density but also their…
In this paper we obtain the central limit theorem for triangular arrays of non-homogeneous Markov chains under a condition imposed to the maximal coefficient of correlation. The proofs are based on martingale techniques and a sharp lower…
We prove a general normal approximation theorem for local graph statistics in the configuration model, together with an explicit bound on the error in the approximation with respect to the Wasserstein metric. Such statistics take the form…
Many machine learning and optimization algorithms can be cast as instances of stochastic approximation (SA). The convergence rate of these algorithms is known to be slow, with the optimal mean squared error (MSE) of order $O(n^{-1})$. In…
In this paper, we refine the Berry-Esseen bounds for the multivariate normal approximation of Polyak-Ruppert averaged iterates arising from the linear stochastic approximation (LSA) algorithm with decreasing step size. We consider the…
In this paper we investigate how standard nonlinear programming algorithms can be used to solve constrained optimization problems in a distributed manner. The optimization setup consists of a set of agents interacting through a…
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm…
The filtering-clustering models, including trend filtering and convex clustering, have become an important source of ideas and modeling tools in machine learning and related fields. The statistical guarantee of optimal solutions in these…